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ABSTRACT 



Three evaluations analyzed program characteristics and 
participant results in work force education and coordination in Washington 
State. The Employment Security Department's evaluation of Job Training 
Partnership Act Titles II and III looked for an association between 
participant characteristics and the type of training they receive and between 
type of training and subsequent employment and earnings. The evaluation found 
that participants with weak academic skills tend to receive remedial 
education, whereas classroom occupational skills training is associated with 
higher wages. The Office of the Superintendent of Public Instruction found 
that, during their first year out of high school, students who had completed 
secondary vocational -technical programs are more likely to work and more 
likely to attend a community or technical college than are other students, 
and they are less likely to take remedial courses in college. The State Board 
for Community and Technical Colleges and the Advisory Council on Adult 
Education reported a joint evaluation on workers upgrading their basic or 
occupational skills at a community or technical college. The key findings are 
that some 24,000 workers take colleges courses to upgrade their job skills 
each year and about 2,500 take basic skills courses. The three evaluations 
are expected to contribute toward building a system of accountability in the 
state job training initiative. (KC) 






* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 







m 

<N 

m 

^*■4 

rj- 



Q 

W 



cr^ 

h 

CSJ 

» 

r- 

'■ © 

ERIC 





WORKFORCE TRAINING 
PROGRAM EVALUATIONS 



1996 



U.S. DEPARTMENT OF EDUCATION 
Olfjee of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 
/ CENTER (ERIC) 

^7 This document has been reproduced as 
received from the person or organization 
originating it. 

□ Minor changes have been made to 

improve reproduction quality. 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL 
HAS BEEN GRANTED BY 

tJ S SSocg 



• Points of view or opinions stated in this TO THE EDUCATIONAL RESOURCES » 

document do not necessarily represent INFORMATION CENTER (ERIC) 

official OERl position or policy. 




2 



WORKFORCE TRAINING AND EDUCATION COORDINATING BOARD 

MEMBERS 

Betty Jane Narver 

Chair 

Gilberto Alaniz 

Representing Targeted Populations 

Judith A. Billings 
Superintendent of Public Instruction 

John Carter 

Labor Representative 

Karen Carter 

Labor Representative 

Roberta Greene 

Business Representative 

Earl Hale 

Executive Director 
State Board for Community 
and Technical Colleges 

Jeff Johnson 

Labor Representative 

Gary Moore 

Commissioner 

Employment Security Department 

Dick Spangler 
Business Representative 

Marian Svinth 
Business Representative 



Ellen O’Brien Saunders 

Executive Director 
Workforce Training and Education 
Coordinating Board 



3 



o 

ERLC 

copy available 



The Workforce Training and Education Coordinating Board does not discriminate or deny 
services on the basis of race, color, national origin, sex, age, religion, or disability. 



Workforce Training 



Program Evaluations 



1996 



Washington State 

Workforce Training and Education Coordinating Board 

Building 17, Airdustrial Park 
P.O. Box 43105 

Olympia, Washington 98504-3105 
Telephone: (360) 753-5662 
Fax: (360) 586-5862 
Internet: wtecb@wln.com 
http://www.wa.gov/wtb 



TABLE OF CONTENTS 



PAGE 

Introduction and Executive Summary : i 

Department of Employment Security 
Job Training Partnership Act 

Program Evaluation A- 1 

Office of the Superintendent of Public Instruction 
Secondary Vocational Education 

Program Evaluation B-l 

State Board for Community and Technical Colleges 
Job Skills Enhancement: 

An Evaluation of Publicly Funded 
Postsecondary Workforce Training 
and State and Federally Funded Basic 
Skills Training for Workers Upgrading 
Their Job Skills 



C-l 



INTRODUCTION AND EXECUTIVE SUMMARY 



Under Substitute Senate Bill 5992 enacted in 1995, the agencies represented on the Workforce 
Training and Education Coordinating Board (WTECB) are to conduct biennial program 
evaluations. These agencies are the Employment Security Department, the Office of the 
Superintendent of Public Instruction (OSPI), and the State Board for Community and 
Technical Colleges (SBCTC). In addition, the Advisory Council on Adult Education is to 
provide a program evaluation of adult basic skills education. This report presents the first 
of these’ evaluations. 

These evaluations analyze particular program characteristics and participant results. The 
research complements "Workforce Training Results," by WTECB. "Workforce Training 
Results" examines the training system as a whole, while the evaluations reported here 
provide more detailed information. 

The program evaluations and "Workforce Training Results" are pieces of an accountability 
system for workforce development, "Performance Management for Continuous 
Improvement" (PMCI). PMCI is a system for setting goals, measuring results, and continuous 
improvement in meeting customer needs. Under PMCI, the purpose of the evaluations is to 
inform policy and program improvements. 

In order to create greater consistency between evaluations conducted by different agencies, 
and as required by statute, WTECB established standards for the program evaluations. The 
standards for the first round of evaluations include: 

1. Data— Agency evaluations should use common data elements and definitions developed 
as part of PMCI.* 

2. Methodology— Agency evaluations should include the relationship between program 
characteristics and post-program results for former participants and include measures of 
program results developed as part of PMCI. 

Agencies should use widely recognized scientific procedures, such as random samples, 
to ensure the validity and reliability of findings, and common survey questions and data 
match techniques, as appropriate, for comparability. 

3. Scope— Agency program evaluation reports should present findings from targeted research 
on sub-components of programs such as fields of study or particular education or training 
strategies that are of interest to them. 

4. Frequency— Agencies should report on program evaluations at least every two years 
(required by SSB 5992). 

5. Use— The purpose of the agency evaluations should be to enable policy improvement. 
Agency evaluation reports, and operating plans and progress reports (required by 



SSB 5992), should include information on how the agency has or will use evaluation 
findings for policy review and program improvement. Operating plans should indicate the 
program evaluations that are planned for the following year. 

6. Format— The reports should be written in narrative accessible to a lay audience and 
include information on: l)the issue or question studied and the approach used to study 
the issue; 2) significant findings with supportive evidence; 3) policy implications; and 
4) an appendix on methodology and other items as appropriate; e.g., data tables and 
survey instruments. 

* Agency program evaluations reported in 1996 are not expected to include specific 
standards such as data definitions or performance measures that are developed after the 
research design has been completed. 

The following are brief descriptions of the three evaluations. 

• Employment Security Department 

The Employment Security Department's evaluation of Job Training Partnership Act 
Titles II and III addresses two questions: 1) "Is there an association between the 
characteristics of the participants (such as education level or prior work history) and the 
type of training they receive? and 2) "Is there an association between the training they 
receive and subsequent employment and earnings?" The Department finds participants 
with weak academic skills tend to receive remedial education, and classroom occupa- 
tional skills training is associated with higher wages. The gain in wages associated with 
skills training is highest for youth participants. 

• Office of the Superintendent of Public Instruction 

OSPI's report evaluates results for students who had completed at least 360 hours (about 
four courses) of secondary vocational-technical education. The evaluation addresses the 
following four questions: 1) "How does the post-high school experience of vocational 
completers compare to other high school graduates?" 2) "How well prepared are these 
students for their next educational experience?" 3) "What programs do these students 
enroll in as they enter community and technical colleges?" and 4) "How should state 
evaluation and information systems be structured so that high quality information is 
available for teachers, parents, and other interested educational audiences?" 

OSPI finds that during their first year out of high school vocational completers are more 
likely to work and more likely to attend a community or technical college than are other 
students. Counting both two- and four-year institutions, vocational completers are just as 
likely to attend college as are other students. Among those in college, vocational 
completers are less likely than other students to take remedial courses in basic skills. 
Among vocational completers attending a two-year college, preliminary results show that 
most enroll in trade and industry programs. The research also indicates the importance of 



continuing to make improvements in the quality and availability of the student data 
maintained by school districts. 

• State Board for Community and Technical Colleges 
and the Advisory Council on Adult Education 

SBCTC and the Advisory Council on Adult Basic Education report a joint evaluation on 
workers upgrading their basic or occupational skills at a community or technical college. 
The key findings are that some 24,000 currently employed workers take college level 
courses to upgrade their job skills each year and about 2,5000 take basic skills courses. 
Most of these workers attend college every few years, attend part-time, and nearly half 
attend at night. In most cases, employers do not pay the cost of the training. Most of the 
workers are satisfied with the quality of training and the related support services, 
although about one-third of the workers are dissatisfied with the availability of classes 
and with advice on course selection. The number of workers enrolled to upgrade job 
skills in college level courses has been declining by five percent per year. In contrast, the 
number of workers enrolled to upgrade basic skills has been increasing by five percent 
per year. 

Beyond their specific findings, the three evaluations will contribute more broadly toward 
building a system of accountability. This is the first time these three agencies have 
simultaneously applied similar research methodologies for the purpose of informing policy 
and program improvement. This experience will provide valuable information as the 
workforce training and education agencies continue to create a system that uses evaluative 
research for continuous improvement. This ongoing effort will build greater data and 
methodological consistency across agencies, and perhaps most importantly, will lead to 
greater application of research for the purpose of informing policy and program decisions. 
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EXECUTIVE SUMMARY 



Background 

Pursuant to RCW 28C. 18.090, the Employment Security Department conducted an evaluation 
of the Job Training Partnership Act's (JTPA) primary program activities. They are Title II-A, 
which serves economically disadvantaged adults; Title II-C, which serves economically 
disadvantaged youth (generally between ages 16 and 21); and Title III, which serves 
dislocated workers (those who lose their job due to plant closure, mass layoff or were 
employed in a declining occupation). 

The goal of the JTPA programs is to improve the employability of participants. In the case of 
the two adult programs, the clear expectation is that they will get the skills they need and get 
a job. In the case of the youth program, the strategy must depend on the circumstances of the 
youth. For those who have not yet graduated from high school, the goal is to help them 
complete their education. For those who have graduated, our expectation is that participants 
either get a job or go on to postsecondary training. 

Although the services vary for each program, all involve three basic kinds of activities: 

1. Skills training— Urns is either offered in a classroom setting, generally through a 
community or technical college, or through an employer; for youth, this is most often a 
work experience in the public sector; for adults, this is typically an on-the-job training 
activity arranged with a private employer. 

2. Remedial education — this is generally for those who have not graduated from high school 
or who have poor English language skills. 

3. Job search assistance — this might be a class in how to look for a job, a job club activity 
with phone banks and peer support, or job development involving a counselor making 
connections with employers on behalf of the participant. 

Research Design 

The design of the evaluation was based on consultation with the 12 service delivery areas that 
actually provide JTPA services locally and staff of the Workforce Training and Education 
Coordinating Board (WTECB). The two primary questions addressed in this report are: 

1 . Is there an association between the characteristics of the participants (such as education 
level or prior work history) and the type of training they receive? 

2. Is there an association between the training they receive and subsequent employment 
and earnings? 
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Findings 

With regard to our first question, we found what we would have expected: namely, that those 
with weak academic skills tended to receive remedial education and the majority in every 
program received some kind of skills training. Although broadly supportive of a relationship 
between participant characteristics and services, the connection was surprisingly weak 
between prior work history and services. While there are a number of possible explanations 
for this, it is worthy of further investigation. 

Our second question is really the heart of this evaluation since it involves relating outcomes 
to what the program did. What we found supported the investment in skills training. 
Although classroom skills training does not always produce rapid employment compared to 
job search assistance and on-the-job training, the skills gained clearly stand up over time, and 
are reflected in higher wages in all three programs. One additional affirmation of training is 
provided by the fact that the degree of earnings gain correlates with the typical skill levels of 
participants in each program. So we see the advantage of skills training highest in the youth 
program and lowest in the dislocated worker program. In other words, the more skills you 
need, the greater your wage gain is likely to be following training. 
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INTRODUCTION 



This study evaluates Washington State's workforce training programs. The focus of this 
evaluation is on JTPA programs. We tracked terminees from program year 1993 (those who 
exited the program between July 1, 1993, and June 30, 1994). Our primary intent is to explore 
the relationship between participant characteristics and the types of services received and the 
relative effectiveness of different services on labor market outcomes. This report was 
intended to provide program operators additional information on the relative merits of the 
services typically provided by JTPA. We hope it will help us develop a consumer report 
system as required by pending federal block grant legislation. 

Research Questions 

In this study, we sought to explore the relationship between participant characteristics and the 
types of services received and the relative effectiveness of different services on post-labor 
market outcomes. 

For example, if we know what level of education an individual has obtained, does that help 
us predict what type of training program(s) he/she is likely to enroll in? 

Similarly, if we know what kind of training service an individual receives, does that help us 
predict his/her employment outcome? 

These and other questions help to formulate the research questions of this study. 

Question 1: Is there an association between participant characteristics (e.g. educational 
attainment, prior work experience) and the type of training service received? 

Question 2\ Is there an association between training services and employment, earnings and 
other related outcomes? 
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I. GENERAL INFORMATION 



A. Categories of Data 

Three major categories of data were used to address the research questions. 

They are: 

1. Services Provided 

For the purpose of this study, we examined the following workforce training 
services typically provided by JTPA: 

• Basic Skills Training (BSK, Basic and Remedial Education) 

• Institutional Skills Training (1ST, Classroom Skills Training) 

• On-the-Job Training (OJT) 

• Work Experience/Entry Employment Experience/Private Internships (WEX 
for Title II only) 

• Other Employment Skills Training (OES, Primarily Job Search Assistance; 
Title II only) 

• Job Search Assistance (JSA, Title II only) 

• Basic Readjustment Services (BRS, job readiness and job search services for 
Title III only) 

Definitions of the training activities are provided in Appendix A. 

2. Participant Characteristics 

The following participant characteristics were used: 

• Sex 

• Race 

• Age . . 

• Educational status 

• Limited English proficiency 

• Prior work history 

3. Outcomes 

Our primary interest is in the program participants' post-labor market outcomes, 
such as employment and earnings. For this evaluation, we examined these 
outcomes in terms of: 

• Employment status of all participants in the third quarter following program 
termination 1 



1 Program termination occurs when the participant leaves the program. They may or may not have 
successfully completed training. 



• Employment status of adult participants at follow-up (thirteenth week 
following program termination) 

• Quarterly earnings of all participants in the third quarter following program 
termination 

• Earnings of adult participants during the follow-up week (thirteenth week 
following program termination) 



B. Data 

Two administrative data files were used in this study: 

• Participant records from the state's JTPA Management Information System 
(MIS) data. 

• Unemployment Insurance (UI) wage records. 

The UI wage file was made available by the Workforce Board, but was collected 
by Battelle. The file contained quarterly data on earnings and hours worked (the 
construction of the quarterly files was discussed in Battelle's report to WTECB: 'An 
Outcomes Evaluation of Washington State's Major Workforce Training Programs'). 
We will consider anyone with reported wages to be employed, and anyone without 
wages to be unemployed. We know we are missing some employment and thus 
overstateing unemployment since not all earnings are covered by UI. 

The JTPA MIS data supplied information on participant characteristics and services 
participants received. It also contained data on the thirteenth week follow-up 
survey; of particular interest to this study were employment status and earnings at 
follow-up. Title II-C Youth participants were excluded from the thirteen week 
follow-up survey because employment is not necessarily the most appropriate 
outcome for them. All JTPA terminees from program year 1993 were identified 
from the JTPA MIS file. 

The original JTPA MIS file for the program year 1993 contained 10,229 records. 

We retained records in Title II-A, Title II-C, and Title III Governor's Reserve and 
substate grantee programs. Records for several smaller programs were excluded, 
including the Older Workers Program, Title III National Reserve Grants, and 
performance incentive awards. The total number of records excluded from the 
original MIS file was 3,488. The remaining 6,741 records were later matched with 
the UI wage file and used for analysis. 



C. Methods 



The results reported here were based on 6,741 records matched with the UI wage 
file. The distributions of the sample across JTPA programs are: 



JTPA PROGRAMS 


MATCHES 


Title ll-A (Adult) 


2,527 


Title ll-C (Youth) 


2,629 


Title III (Governor’s Reserve & Substate 


1,585 


Grantee Programs) 




Total 


6,741 



A 



The thirteenth week follow-up sample was a statistically valid sub-sample of the 
6,741 observations used for analysis. It was comprised of: 



PROGRAMS 


SAMPLE 


Title ll-A (Adult) 


1 ,474 


Title III 


927 


Total 


2,401 



The primary purpose of the thirteen-week follow-up is for calculating Department 
of Labor performance standards. There is no follow-up performance standard for 
youth because employment is not always an appropriate outcome. The youth 
performance standards are based on data at the time youth leave the program. 

Of the 6,741 observations, 3,433 individuals were involved in a single training 
activity, and 3,319 were reported to have received multiple services. The 
distribution of training services by each program is presented in Table 1. 

Two sets of analyses were conducted: 

• Cross-tabulations (to display the association between two variables) 

• Chi-square tests of independence (test of statistical significance in the 
association) 

Results presented in this report were statistically significant (p<0.05) unless 
specified otherwise. 

We analyzed participant characteristics and outcomes based on program service 
types. The results showed that about half of the sample received more than one 
service. The procedures to handle the multiple service participation were that we 
first grouped services into discrete categories, and assigned each participant to one 
of those categories. 
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In the process of developing methods to classify service clusters we found that some 
clusters had very few participants. This is particularly true with the JTPA Title III 
sample. In addition, the patterns of cluster sample sizes varied from program to 
program. 

To overcome this problem, we took into account the sample composition and 
sample size when we combined services. We also made sure that each combination 
was meaningful, i.e., grouping similar programs together. In doing so, we treated 
Title II-A, II-C, and Title III samples separately according to the cluster sizes and 
training types. This procedure resulted in seven categories for Title II-A, eight for 
Title II-C and four for Title III program. All our analyses were based on these 
service clusters. 

D. Definitions 

The following are the definitions of service clusters by program title. 

Title n-A 

1 . Basic Skills Training (BSK, Basic and Remedial Education) 

2. Institutional Skills Training (1ST) 

3. On-the-Job Training (OJT, including WEX, very few in number) 

4. Job Search Assistance (JSA/OES) 

5. Training (mostly IST)+ Job Search Assistance 

6. Multiple Training components (mostly a combination of 1ST and WEX) 

7. Remedial Education (BSK) + Training (mostly 1ST) 

Title n-C 

1 . Basic Skills Training (BSK, Basic and Remedial Education) 

2. Institutional Skill Training (1ST) 

3. Work Experience (WEX, including OJT, very few in number) 

4. Job Search Assistance (JSA/OES) 

5. Training (mostly WEX) + Job Search Assistance (JSA/OES) 

6. Multiple Training components (mostly a combination of 1ST and WEX) 

7. Remedial Education (BSK) + Training (mostly WEX) 

8. Remedial Education (BSK) + Job Search Assistance (JSA or OES) 

Title m 

1 . Basic Skills Training (BSK only, or in combination with another service 
component) 

2. Institutional Skills Training (1ST) 

3. Institutional Skills Training (IST)+ Basic Readjustment Services (BRS) 

4. Basic Readjustment Services (BRS) only 
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Although participants can only be in one service cluster for purposes of analysis, we 
must add clusters to get the total of those receiving a particular kind of training. For 
instance, the total of youth receiving WEX is the total number of people in clusters 
3, 5, 6, and 7 above. 



II. FINDINGS-TITLE ll-A: ADULT PARTICIPANTS 

The results presented below were based on 2,527 matched observations with the UI 
wage file. Although the sample was found to be in a variety of training components, 
eighty-two percent received some kind of skills training with enough people getting 
1ST and OJT that we could look at the outcomes for each separately. In fact, if you 
include basic or remedial education, over 95 percent of the participants received 
education or training. 

A. Participant Characteristics and Services Received 

Table 2 presents the results of our analysis of the relationship between the 
participants’ characteristics and the services received. 

1. Gender and Services Received 

The JTPA MIS file for PY 1993 terminees shows that JTPA II-A programs 
were largely comprised of female participants (67 percent). While females 
outnumbered males in all components, they were far more likely to receive their 
training in a classroom setting. Approximately two thirds of females received 
1ST, either alone or in combination with some other activity. For males the 
comparable figure was not quite a half; instead, males were more likely to get 
training. on the job (37.5 percent vs 27.8 percent). Although both genders were 
equally likely to take BSK, females more often added a skills training activity 
than did males. Overall then, it appears that women were proportionately more 
likely to receive more intensive services involving multiple components (37.6 
percent versus 26.2 percent). 

2. Race and Services Received 

Of the 2,527 records, 1,635 were White. The sample distribution among minority 
groups were: 302 Black, 293 Hispanic, 108 American Indians, and 189 Asian or 
Pacific Islander. 

Table 2 indicates that people with different ethnic backgrounds had different 
training needs. 

• The pattern of training for Whites and Native Americans was roughly the 
same, both getting classroom training around 63 percent of the time, and OJT 
over 20 percent of the time. 

0 
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• Over 70 percent of Blacks received classroom training, higher than any other 
group. 

• As might be expected, Asians were particularly apt to receive remedial 
education (57.2 percent), with Hispanics next at 41 percent. 

3. Education and Services Received 

Among those who did not complete a high school education, over half received 
Remedial Education, either alone (34.6 percent) or in combination with 
classroom training (20.7 percent). The only surprise in this area is that the better 
educated (those with some college or a BA) were not more highly represented in 
the small number of participants who received only job search assistance. 

4. Limited English Proficiency and Services Received 

About 10 percent of the sample were reported to have limited English 
proficiency. Results from Table 2 reveal, as expected, almost 60 percent of the 
individuals with limited English proficiency received Remedial Education, 
which includes English-as-a-second-language classes. More surprisingly, they 
were slightly more likely to receive an OJT than those without an English 
language deficiency. This may indicate a willingness on the part of some 
employers to overlook this “barrier.” 

5. Prior Work History and Services Received 

Table 3 presents results between prior work history and services received. 

As noted earlier in the report, two indicators were used to measure prior 
work history: 

• Lacks significant work history (Yes/No) 

• Number of weeks unemployed (ranging from ‘0 week’ to ‘26 weeks’) 

The results showed that 44 percent of the sample (1,113) had not worked for 
the same employer for longer than three consecutive months in the previous 
two years. And 42 percent had been unemployed for 26 or more weeks. 
Surprisingly, the differences in the mix of services received did not seem 
much affected by prior work history. Additional research will be necessary 
to understand why there is not a stronger relationship here. 

B. Service Types and Post-Labor Market Outcomes 

Table 4 enables the reader to see the relationship between the kind of service 
received and the subsequent employment. This is really the heart of this report. As 
noted earlier in the report, we measured post-labor market outcomes in terms of: 



• The third quarter employment status— this was measured by looking at employer 
records of wages covered by UI. It does not capture “uncovered” employment, 
such as self-employment. 

• Employment status at follow-up-this is done by a direct contact with a sample 
of former participants. 

• Average quarterly earnings in the third complete quarter following program 
termination. 

• Average weekly earnings during the thirteenth week following program 
termination (follow-up survey). 

C. Employment Status 

About 72 percent (1,474) were reported to be employed one quarter following 
termination, going down to 62 percent (2,527) in the third quarter following program 
termination. 

Individuals enrolled in Remedial Education were least likely to be employed later 
(59.9 percent employed at one quarter and 53.5 percent at three quarters); however, 
they could increase their likelihood of being employed by 10 percent by including 
skills training in the mix of services received. 

All other services, including job search assistance, produced relatively consistent 
results, ranging between 69.7 percent for classroom training and 76.5 percent for 
job search assistance at one quarter (the comparable figures for three quarters after 
termination were 60.2 percent and 63.7 percent). However, after three quarters, the 
added investment in multiple services became apparent. For example, 71 percent 
of those receiving two kinds of training plus job search assistance were still 
employed, showing very little drop from their employment at one quarter (74.7 
percent). In other words, the additional investment in training is showing a payoff 
in job retention. 

D. Earnings 

Consistent with the employment outcomes, earnings were measured at two points 
in time: 

• Average earnings at follow-up (thirteenth week following termination, reported 
as weekly earnings) 

• The average earnings in the third quarter following program termination 
(reported as quarterly earnings) 

The average weekly earnings in the follow-up week were $270. The average 
quarterly earnings in the third quarter were $3,095 (or about $238 per week). These 
results are contained in Table 4-A. 

20 
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In contrast to the results on employment where classroom training fared less well 
than other types of training, exactly the reverse is true with earnings. The earnings 
for the classroom training group were $292 at 13 weeks and $3,456 at three quarters 
following termination. 

The policy implications of these findings are difficult to sort out. While the least 
expensive service (job search assistance) fares well in these data, it is presumably a 
reflection of the skills those participants bring to the program. Similarly, the 
relatively poor performance by those receiving remedial education is almost 
certainly explained by the additional educational barriers they faced. 

It appears safe to conclude that additional training (regardless of the kind of 
training) pays off in a greater chance for employment and in higher earnings once 
employed. And the differences between kinds of training balance out when you 
look both at the likelihood of being employed and at subsequent earnings. 



III. FINDINGS-TITLE ll-C: YOUTH PARTICIPANTS 
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A. Participant Characteristics 

Table 5 displays service distributions by participant characteristics. The results 
presented below were based on 2,629 case files, matched with the UI administrative 
wage file. 

1. Participant Characteristics and Service Types 

Because JTPA emphasizes serving "at risk" youth, and because one criteria of 
being at risk is being behind in school, it is not surprising that over half receive 
remedial education (53.4 percent), mostly in combination with some other kind 
of training. One difference between Title II- Youth and Adults in their respective 
service strategy is that youth were overwhelmingly directed toward WEX rather 
than OJT or towards public sector training opportunities. 

This reflects a different approach to working with employers. In the case of 
youth, the lack of skills represents a deterrent to employers, which leads to a 
strategy of giving youth work experience in the public sector where no wages 
are paid by the employer. This strategy is much more suitable to those without 
prior work history, while the reverse is true for OJT. 

2. Gender and Services Received 

Title II-C was closer to en even split, at 46 percent for male and 54 percent for 
female. Overall, gender was not found to be related to the kinds of services 
received. This is a positive finding, given the persistent tendency in the overall 
economy for males to earn more than females. 
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3. Race and Services Received 



The pattern described above for adults does not repeat itself in the youth 
program. In fact, the differences that exist tend to be relatively small. 

• Black Americans were more likely to receive remedial education without 
subsequent training than other ethnic groups (22.8 percent compared with 
the average of 1 5.6 percent). 

• Both Blacks and Asians were slightly under-represented in the multiple 
training category (50.4 percent for Blacks and 53.7 percent for Asians 
compared to an average of 60.9 percent). Neither finding is likely to be 
significant given the fact that many youth are not yet in the labor market. In 
any case, the subsequent employment and earnings results do not suggest the 
differences are likely to be important. 

4. Age and Services Received 

The data divided youth into those below 18 and those 18-21 . Once again, the 
most surprising finding was that there were no dramatic differences in the kinds 
of services received based on age alone (see Table 5). The differences we did 
see were what was to be expected. 

• People under 18 were more likely to receive Remedial Education (29.3 
percent versus 21.3 percent). 

• In contrast, the older group were more represented in Classroom Skills 
Training (16.2 percent versus 4.3 percent), although this was not a common 
strategy in either group. 

5. Educational Attainment and Services Received 

The results (Table 5) show that 75 percent of the sample did not complete high 
school. We found relatively small differences in the kinds of services received 
between individuals with a high school diploma and those without it: 

• Sixty percent of individuals with less than a high school education received 
remedial education, either alone or in combination with some other kind of 
training, compared to forty-eight percent for high school graduates. 

• Those with at least a high school diploma were most likely to receive 
training (68.5 percent in WEX and 35.4 percent in 1ST, including some in 
both). The comparable numbers for the non-graduates were 63.7 percent 
in WEX and 4.8 percent in 1ST. Here again, this is what we would expect. 
The apparent preference for employer based training (WEX) among youth 
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over classroom based training may partly be explained by the opportunity 
to receive compensation for the work done. However, it also undoubtedly 
reflects differences in maturity and skill levels compared to the adult 
population, resulting in youth getting a subsidized and structured work 
setting. 

B. Service Types and Post-Labor Market Outcomes 

1. Employment Status 

For this sample, only the employment status in the third quarter following 
termination was used because the thirteen-week follow-up survey did not 
include youth. Table 4 presents employment outcomes across service categories. 

Because many youth are returning to school rather than looking for work, 
looking at employment rates is misleading. With that caveat, it is interesting 
to contrast the results for employment and earnings of youth with those of 
the adults. 

Contrary to the adults, employment rate for Classroom Skills Training was 
62 percent, appreciably higher than any other kind of training. By contrast, 
job search assistance was the least successful at only 41.2 percent (when 
not coupled with other training) followed closely by remedial education at 
42.3 percent. 

This finding would have greater policy significance if large numbers of youth 
received job search only, but they did not. Only five percent of the youth sample 
received this kind of training. It does suggest employers place a high weight on 
job skills for youth since few have any significant work history. 

2. Service Types and Earnings 

• The average quarterly earnings for the Title II-C Youth sample were $1,741 
(Table 4-A), significantly lower than that of the adult sample. This reflects 
not only a lack of prior work experience, but also a higher percentage of 
part-time employment. Unfortunately, we do not have data to verify the 
latter point. 

In the case of earnings, the results confirm the findings of the adult program. Once again, it 
is Classroom Skills Training that produces the best earnings. In fact, at $2,700 per quarter, it 
is almost $500 higher than the next highest service cluster, which is Classroom Skills 
Training combined with other training. And, it is fully $1,000 higher than the highest service 
cluster that does not include Classroom Skills Training as a component. 
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IV. FINDINGS--TITLE III PARTICIPANTS: DISLOCATED WORKERS 

Because very few individuals in this sample received OJT (WEX services are not 
allowable), we were able to combine all types of skills training into a single category 
of service. Since the majority of them received classroom skills training, we will use 
the label Institutional Skills Training to apply to the group. 

The results presented below were based on 1,585 participant records, matched with the 
UI wage administrative file. The sample includes both those in the Governor's Reserve 
program and those in the substate grantee program. 

A. Participant Characteristics and Services Received 

The results discussed below are shown in Table 8. 

1. Gender and Services Received 

About two thirds of the Title III sample (6 1 percent) were males, reversing the 
gender distribution for the Title II-Adult sample (Table 2). Over half the women 
received a combination of skills training and readjustment services (or job 
search assistance). This is the same pattern as we saw in Title II-A, suggesting 
women tend to get more intensive services than men. 

2. Race and Services Received 

Because this population is almost 90 percent White, there is little point in 
looking at the other race categories separately. And taken as a whole, they 
reflect a mix of services very similar to that of Whites. 

3. Educational Attainment and Services Received 

Clearly, the Title III population is much better educated than their counter- 
parts in Title II. 39.5 percent have more than a high school level education, 
while fewer than 10 percent have less than a high school degree. For those 
with at least a high school diploma, there are no substantial differences in 
the service mix. 

4. Prior Work History and Services Received 

Because dislocated workers must have been employed to qualify, it is not 
surprising that only two percent (eleven) of the sample lacked a significant work 
history. Looking at weeks unemployed, there are essentially no differences in 
how participants were served. 
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B. Service Types and Post-Labor Market Outcomes 

1. Service Types and Employment Status 

• Looking at Table 10, we get yet a third view of the relationship between 
service and employment. In this case, the advantage once again goes to 
skills training (80.7 percent employed at 13 weeks and 75.2 percent with 
covered wages after 3 quarters). Other services were not far behind for both 
follow-up periods. 

2. Service Types and Earnings 

It is in the area of earnings that this population reflects the advantages of their 
education and work history. The average wages are substantially higher than 
Title II adults, even for those in the least successful category of remedial 
education. The average quarterly earnings after nine months were $5,060. The 
weekly earnings at the thirteenth week were $419. 

This study shows that it is skills training that produces the highest earnings at 
$5,507 for the third quarter and $477 for the thirteenth week. (Table 4-B). But, 
because of the significant work experience of this group, even those that need 
basic education receive higher wages than the Title II-A service component with 
the highest wages ($4,568 versus $3,456). 



CONCLUSION 



Title II-A 



Approximately two thirds of these disadvantaged adults are female, and 56 percent lack any 
significant work history. Further, 21 percent have not completed high school. 

The services to respond to this population typically consisted of training (over 80 percent 
received some form of training). This occurred either in a classroom setting (most often 
provided by a community college or vocational technical college) or in a work setting 
(provided by the employer). There were some surprises in looking at who got what service. 
For instance, 45 percent of those that had not completed high school did not get basic skills 
training. Also, there did not appear to be a strong relationship between a participant’s prior 
work experience and the kind of training they received from JTPA. 

Title H-C 



This is a program for disadvantaged youth. The majority are female, and 75 percent have 
not completed high school; just over half are eighteen years old or older. The percentage 
of minorities (just over 40 percent) is higher in this program than either of the JTPA adult 
programs. 

The two primary services are: 

• work experience, which involves placing a youth in a public sector job in a setting where 
skill development can occur. 

• basic skills training, which is academic training designed to return students to the grade 
level of their peers. 

Of course, for those ready to go into unsubsidized work, there is job search assistance. The 
key to a quality program is linking the education and training together. An even stronger 
linkage can be achieved by tying the two activities to the participant’s academic or in-school 
work. 

t 

Title m 

The data clearly reflect the very different nature of the dislocated worker population. It is 
far more representative of the working population as a whole. It is the only JTPA program 
serving a majority of males. They are older and less likely to be minorities with much fewer 
of the barriers to employment that characterize the Title II programs. 

Overall Outcomes 
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It is not surprising then, that the services they receive reflect this difference. This population 
needs skills upgrading or retraining, and help finding a new job. They have already proven 
their value as workers. They are looking for an opportunity to return to work. However, they 
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have a special problem— they have lost a job where they had seniority and a relatively high 
wage. For this group, getting back to that wage will be a challenge, even with training. 
Nevertheless, the wages they command are far higher than the Title II adults ($5,060 per 
quarter versus $3,095). 

Looking broadly at the services provided and the kinds of individuals in the three programs, 
the fit was pretty good. The more disadvantaged the population, the more likely they were to 
get remedial education. Conversely, those with the most education and training received 
more assistance with their job search. This is illustrated in the chart below. 





Title ll-C 
Youth 


Title ll-A 
Adult 


Title III 

Dislocated Workers 


Basic Education 


53.4 


25.5 


5.2 


Training 


69.8 


82.2 


68.5 


Job Search 


37.6 


26.1 


72.9 



Note: Totals add to more than 100 percent because many participants received more than 
one kind of service. 

In terms of the outcomes, there is clear support for the value of skills training provided by 
our community and technical colleges, as well as private colleges. The training is valued by 
employers as evidenced by the wages. This is particularly impressive when you compare the 
wages of classroom-based training with that provided by the employers themselves (OJT). 
Although they are training for different jobs, it is interesting that the wages for Classroom 
Skills Training are still higher nine months later. 

Determining what program changes should be made based on these data is more difficult 
for the youth program and for dislocated workers. Because the design of youth programs 
has been evolving in the direction of a program with goals balanced between education and 
employment, there has been an increasing focus on a stronger educational component since 
program year 1993. In the case of Title III, there are such crucial differences in the industries 
affected from one year to the next that it is less clear whether the lessons learned from one 
industry would apply to another industry. The differences in education and the transferability 
of skills of an ex-Boeing worker and a timber worker is likely to be dramatic. 



Appendix A: 

DEFINITIONS OF TRAINING ACTIVITIES 

BSK- Basic Skills Training : Instruction normally conducted in an institutional setting 
and designed to upgrade basic skills prepare the individual for further training, future 
employment, or retention in present employment. It includes remedial reading, writing, 
mathematics, literacy training, study skills, English for non-English speakers, bilingual 
training, GED preparation (including computer-aissisted competency training, and school 
to postsecondary education transition). 

1ST- Institutional Skills Training : Instruction conducted in an institutional or work site 
setting designed to provide or upgrade individuals with the technical skills and information 
required to perform a specific job or group of jobs such as auto mechanics, health services, 
or clerical training. It includes job specific school-to-work/apprenticeship programs, on-site 
industry specific training, customized training, entrepreneurial training, internships, and 
pre-apprenticeship training. 

OJT- On-The-Job Training'. Training in the public or private sector which is given to an 
individual while she/he is engaged in productive work and which provides knowledge or 
skills essential to the full and adequate performance of the job. 

WEX- Work Experience - Entry Employment Experience/Private Internships : Work 
Experience is a short-term or part-time work activity in the public or not-for-profit sector, 
which provides an individual with the opportunity to acquire the skills and knowledge 
necessary to perform a job, including appropriate work habits and behaviors. For youth only, 
Entry Employment Experience or Private Internships is a formal opportunity to examine or 
investigate employment typically at private, for-profit work sites. 

OES - Other Employment Skills Training : (Title II) It includes activities such as pre- 
employment/work maturity training and non-job-specific school-to-work/postsecondary 
programs (does not include job search assistance or basic readjustment services). 

JSA - Job Search Assistance: (Title II only) A service that helps a participant seek, locate, 
apply for, and obtain a job. It may include job finding skills, orientation to the labor market, 
resume preparation assistance, job development, referrals to job opening, job clubs, 
vocational exploration, and relocation assistance. 

BRS - Basic Readjustment Service: (Title III only) It includes services designed to provide 
basic readjustment assistance such as orientation, skills determination, pre-layoff assistance, 
job development/referral assistance, and job search to eligible dislocated workers. 



28 

ERIC 



A-18 



Appendix B: 

VARIABLES AND DEFINITIONS 

Age — Age contained the following categories: 

Under 18 18-21 

22-29 30-54 

55 and over 

Educational Status — The original definition was 'Highest school grade completed.' For the 
purpose of our analysis, we reconstructed the variable into the following groupings: 

'No school grade completed' 'Less than high school 

'High school graduate' 'Some college' 

'Bachelors or beyond' 

Prior Work History— Two variables were identified from the JTPA MIS file: 

1 . Lacks significant work history— An individual is reported to lack significant work 
history if she/he had not worked for the same employer for longer than three consecutive 
months in the two years prior to application. 

2. Number of weeks unemployed during the prior 26 weeks— The original coding for 
‘number of weeks unemployed...' ranged from 0 week to 26 weeks. We recoded the 
variable into the following discrete categories: 

*Never unemployed' 'Unemployed 1-6 weeks 

'Unemployed 7-12 weeks' 'Unemployed 13-18 weeks 

'Unemployed 1 9-25 weeks' 'Unemployed 26 weeks' 

Third quarter employment status following program termination-the UI wage file did not 
have a data element that could be used directly to measure quarterly employment. We then 
used the quarterly wage information to construct the variable. An individual was recorded 
'employed' if the UI covered employment in that quarter showed earnings greater than zero. 
Otherwise, the individual was recorded 'unemployed.' 

Youth competencies attained— the following data elements were extracted as indicators of 
youth competencies attained from the JTPA MIS file. All these variables were recorded as: 

'l=Yes' and '2=No': 

Youth Employability Enhancement Terminations ('Yes’ or ‘No') 

Attained Preemployment/ Youth Employment Competency 
Attained Basic Education 
Attained Institutional Skills/Job Specific Skills 
Completed Major Level of Education 
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INTRODUCTION 



This evaluation study used the information collected by a graduate follow-up feasibility study 
to address an evaluation mandate for operating agencies of the state training system called for 
in Substitute Senate Bill 5992. This study created outcome measures based on student 
experiences in their first year after high school. Data matches were completed one full year 
after the end of the 1993-94 school year. The evaluation questions addressed the preparation 
of students for their next educational experience and the type of programs these students 
enroll in at community or technical colleges. Information was also gathered regarding the 
need for remediation in basic skill areas for students who had completed a vocational 
program of study . 

The methods used in this study were successful in locating 93 percent of the students in 
participating districts who were identified as completers. Over 40 percent of the students 
located went to work without taking additional training. Thirty-six percent of these students 
enrolled in a community or technical college, and 13 percent were attending a four-year 
college during their first year after graduating from high school. More than one half of the 
students who supplied program information at the community college they were attending 
had enrolled in trade and industry courses. 

The study also successfully addressed questions about evaluation methodology. While there 
were limitations to this study based on a lack of complete data, procedures will be put into 
place during a second year that would address these deficiencies. It is our recommendation to 
continue to use the graduate follow-up methodology to address evaluation questions about 
vocational students. The quality of the information will continue to improve as districts 
become more familiar with the information they need to collect and as trend information 
becomes available over the next few years. 



EVALUATION QUESTIONS STUDIED 

The information collected from the Graduate Follow-up Study described below was analyzed 
to address three evaluation questions that are important in understanding the effectiveness of 
vocational education for students who complete 360 hours (about four courses) in a particular 
occupational or skills emphasis. These questions were addressed as part of a secondary 
analysis of the data collected from participating districts in a voluntary feasibility study that 
has been conducted for the past three years. These questions were: 

• How does the post-high school experience of vocational completers compare to other 
high school graduates? 

• How well prepared are these students for their next educational experience? 

• What programs do these students enroll in as they enter community and technical 
colleges? 
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Another question is particularly important as consideration is made of the type of information 
necessary to answer questions about how to successfully prepare students for the working 
environment of the next century. 

• How should state evaluation and information systems be structured so that high quality 
information is available for teachers, parents, and other interested educational audiences? 



DESCRIPTION OF THE PROCEDURES FOLLOWED 

Past graduate follow-up studies have been based on surveys sent to students from each 
graduating class. In these surveys students were commonly asked whether they were 
currently attending college, whether they went directly to work after high school and how 
successful their high school was in preparing them for this important next step in their life. 
These studies were characterized by high costs, low rates of return, and over-reporting of 
college attendance. In this study the primary question of how well high schools prepare 
students for their next step was approached in a quite different way. A student roster was 
extracted from each district’s computerized student record system. A social security number 
(SSAN) was available in most local school districts, and this unique student identifier 
provided the basis for an electronic match against the student record systems at colleges, 
universities, and the Employment Security Department. 

By joining files from each of these sources for the graduating class of 1994, it was possible to 
produce one database with a wealth of information about students' next step after high school 
and, at least initially, how well they did once they got there. For the third year participating 
school districts were able to supply student data as a fixed or delimited ASCII file in PC 
compatible format. Although each district has its own student record format, field layouts 
supplied by each district resulted in information being readily interpreted and easily added to 
a cumulative database. Recent versions of databases and statistical packages have become 
very powerful in their search, classification, analysis, and graphical capabilities. These 
advances play an important role in the thoroughness and economy necessary to analyze the 
massive amounts of data involved. 



SIGNIFICANT FINDINGS 

The primary purpose of the Graduate-Follow-up Study is to return information to each 
participating district and high school about the success of the students in each graduating 
class. By gathering information using the procedures described above, it was possible to 
create a set of quality indicators that will help schools consider how successfully students are 
prepared for their first educational or work experience after high school. System indicators 
are being developed for schools in many states as a part of the new assessment system called 
for in state and national reform initiatives. The Graduate Follow-up Study is one attempt to 
develop indicators of school quality and reinforces the work being done at the school, district, 
and state level in Washington and in other states across the nation. 
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Presented below is a set of the preliminary indicators for the 4,454 students reported as 
vocational completers in the districts that participated in our voluntary feasibility study'. 
These students comprise about one fourth of the 19,178 students in participating districts 
who graduated with the class of 1994. 

Thirty-one districts participated in this study representing all regions of our state. Large 
and small districts were involved as were two of the occupational skills centers. It is 
important to point out that while nearly a third of the students in the state who graduated 
with the class of 1 994 were included in this study, the data cannot be considered technically 
representative of all students' experiences since districts participated on a voluntary basis. 
Despite this limitation, the size of the sample makes the data of interest and, to the 
researcher's knowledge, this represents the most complete information collected in the 
state. Plans are in place to address the limitations of this sample as part of the study of the 
class of 1995. 



RESULTS 

Table (on page 4) presents a graphic that describes the post-high school experience of 
students during their first year out of high school. It is from this information that program 
evaluators can begin to create quality measures for schools as well as provide accurate 
information about how successful programs and practices are in preparing these students. 

Some examples: 

• Of the vocational completers who supplied a social security number, 93 percent were 
located. This percentage was higher than the percentage for the total student sample 
where about 8 1 percent of the students were located. Many students had engaged in more 
than one activity during their first year after graduation. For example, more than 22 
percent of vocational completers enrolled in a community college and worked during 
the preceding year. 

• Table 1 provides a graphic display of the student sample after those without SSANs 
were excluded. This represents a best estimate of percentages of students in each 
category. About 42 percent of the students classified as vocational completers were 
employed only during their first year out of high school. This compares to 26 percent 
of all students in the sample. 

• Table 1 also presents information about students who went to college at a four-year . 
university, a community or technical college in the State of Washington. Nearly 36 
percent of vocational students went to a community or technical college compared to 
slightly less than 30 percent of all students. A smaller proportion of vocational students 
went to four-year colleges than was the case for all students (12.7 percent vs 16.7 percent) 



Table 1 

Post High School Efforts of Vocational Students 




Students without social security numbers have been 
excluded. Students employed and attending college were 
considered college only. Students attending two and four year 
college were considered four-year college only. 



REMEDIAL NEEDS 

• Vocational students who attend college are less likely to need remedial work in English 
and math than are other students (Table 2). Slightly less than 19 percent of the vocational 
students were required to take remedial courses in basic skills after enrolling in college 
compared to more than 23 percent of all students. Nearly nine percent of vocational 
students required remediation in math, slightly more than five percent in English and 
about four percent needed help in both areas. More females needed help in math, and 
more males needed assistance in English. This information will need some refinements 
as this study expands. Not all four-year colleges supply remedial information. One 
institution sends students with remedial needs to community colleges. One institution 
reports no students with remedial needs. Whether all technical programs require the same 
entry level in basic skills as universities also needs to be determined. 
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Table 2 

Remedial Coursework for 1994 Vocational Graduates 



No Remedial Work 
81 % 




S Remedial Math 

Remedial English 9% 
Remed. Math & 6% 



Engl. 

4% 



Not all four-year colleges provide remedial information. 



Additional information is available to provide perspectives on the performance of various 
subgroups of students. These data contain important information about the extent to which 
we are achieving equity in our educational programs and practices. 

Declared Program Enrollment. The third evaluation question requested information about 
the declared program enrollment of vocational students at the community and technical 
colleges in our state. Table 3 presents the information we were able to produce regarding 
this question. Several issues need to be addressed to set the context for this information. We 
are confident this question can be addressed through this kind of follow-up study. We are 
equally certain that the information presented here should be considered preliminary and 
perhaps would be best used to determine what steps are necessary to collect more complete 
information in a subsequent study. The information presented is for those districts that 
participated and were able to provide information to us about their vocational completers. 
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Table 3 

Occupational Areas of Study at Community Colleges 



. Early Childhood 

Business Education Agnculture Education 

13 % 1/0 12 % 




58 % 



• One percent of the students enrolled in agriculture programs at the community college. 

• Early childhood education courses attracted 12 percent of the students. 

• Trade and industry was the largest category with 58 percent of the reporting students 
enrolled. 

• Health occupations 1 5 percent 

• Business education 13 percent 

• The remaining students who reported a program code were in other categories. 



SUMMARY AND CONCLUSIONS 

The information presented in this paper comes from two research projects conducted during 
the past year as part of the Graduate Follow-up Study. The initial study conducted with 22 
volunteer districts was supplemented by a study that added nine more districts and was 
conducted with Battelle under contract with the Workforce Training and Education 
Coordinating Board. For the purposes of this evaluation report, these two separate data sets 
were combined to address the evaluation questions presented in the first section of this paper. 
This process involved some risks in evaluation methodology that need to be addressed in 



subsequent evaluations. As agency staff work with high schools and school districts across 
the state, the completeness and accuracy of this information will improve. The study for 
the class of 1996 will offer this technology-based methodology to every high school in 
Washington. As districts become more familiar with computer technology and as it is 
demonstrated that this method can replace the multiple reporting processes now required 
of schools, dramatic improvements can be made in the quality of the information available 
for the decision making process at the school, district, state, and legislative levels. 

The most basic principle in program evaluation is to define the evaluation questions before 
preparing the design and data collection procedures that will be used. This principle was not 
adhered to here, a not uncommon experience for evaluators. In future studies, work needs to 
continue in defining the evaluation questions from each educational audience. In this way, 
information gathering processes will be refined and produce the kind of information 
necessary for sound educational decision making. 



RECOMMENDATIONS FOR PROGRAM AND 
MANAGEMENT IMPROVEMENT 

The ability to locate, through data matches, a large number of students who had completed at 
least 360 hours of instruction in secondary vocational education programs was positive. That 
93 percent of these persons were employed and/or engaged in a higher education program at 
the two-year or four-year level was a finding school personnel and community members can 
use to reinforce and improve school-to-career offerings and strategies which prepare students 
for work and further education. 

The feasibility of doing follow-up of graduating classes, including vocational completers, 
has been tested over the past three years with additional schools participating each year. 

It is believed that this method of evaluating students post-high school activities and 
accomplishments will be more cost effective and give schools data needed to evaluate the 
outcomes of their program efforts for all students. 

Technical assistance from this agency is needed to encourage and assist school district 
personnel to maintain the common data elements needed and to handle various coding issues 
in a consistent manner, in order to provide quality information as a tool for program 
evaluation and continuous improvement. Having consistent data from a greater number of 
schools will allow analysis of additional subsets of participants such as gender and ethnicity, 
relationship between training and further education and/or employment, and levels of job 
types in working toward the goal of high skills/high wages 
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EXECUTIVE SUMMARY 



Evaluation Issues and Approaches 

This study meets Senate Bill 5992 requirements for biennial program evaluation by the 
Washington State Board for Community and Technical Colleges (SBCTC) and the state’s 
adult basic education providers. 

SBCTC chose to research the job skills enhancement training area because service to this 
population is an important mission of the community and technical college system. It is an 
area which has received less attention than job preparatory training or retraining (HB 1988). 

Data sources for this analysis include: 

• A survey of 1,151 students at eight representative colleges was completed during 
November 1990. The survey included workers enrolled in developmental and college- 
level courses (Nolte). 

• Focus group interviews with workers enrolled at six community colleges in spring 1990 
(Borton). 

• Data matches with the Unemployment Insurance system wage records for the 13,200 
basic skills and college-level students enrolled for job skills enhancement who left their 
college in 1991-92 (Simmons). 

• College records of the approximately 27,000 workers enrolled to upgrade job skills each 
fall quarter from fall 1990 to fall 1995. 

Key Findings 

• Job skills enhancement students typically enroll in state-supported courses to meet their 
upgrading needs. Their approach to the college is much like a “commodity” purchase. 
That is, they enroll if the college has a product which they need at the time. The majority 
have used the college every few years to meet their upgrading needs. Most enroll part- 
time with nearly half taking classes only at night. The majority enroll without the direct 
support of employers. Just 27 received tuition reimbursement. Eleven percent enrolled in 
employer contracted courses. 

• The vast majority of workers (88 percent) enroll in classes funded by tuition and state 
resources, not in employer funded courses. 

• Workers take courses from across the whole curriculum including math, English, 
psychology, and foreign language, as well as occupationally specific courses. 

• Workers were satisfied with the quality of training and the resources provided to support 
training. 



• Some 35 percent of workers were dissatisfied with the availability of classes they need for 
upgrading. While the majority were satisfied with advising assistance related to course 
selection, 30 percent see a need for improvement in that area as well 

• The number of workers enrolled in job skill enhancement college-level classes has 
declined by five percent a year since the beginning of this decade. As of fall quarter 1995, 
some 24,000 workers are taking college-level courses to upgrade their job skills. 

• The number of workers enrolled in job skills enhancement at the pre-college level has 
grown by five percent a year, but there still is a need to increase service. Basic skills 
enrollment of workers represents 10 percent of the upgrade training— about 2,500 students 
in colleges. Currently a unit record database does not exist to provide a similar count for 
community-based organizations which provide basic skills training. The State Literacy 
Survey (SALS) found that 57,000 employed adults needed such training as they scored at 
the first or second skill level in the SALS. This difference between the number now 
enrolled and the need represents a substantial opportunity to increase service. 

• The typical $1 1 an hour post-training wage earned by workers upgrading at the college- 
level and the $8 an hour at the basic skills level cannot be attributed as a value-added by 
the program. These post-training wages reflect background characteristics of students 
rather than the impact of recent training at the college. For this reason, wage data, though 
included in the report, are not, presented as a measure of the programs outcomes. 

Policy Implications 

Job skills enhancement students, especially those who enroll in college-level courses, 
represent life long learners who use the college and other training resources as a means to 
increase their satisfaction with their own work performance or increase their opportunities for 
better work assignments. Workers rate their college experience positively. 

The state can assure that colleges can effectively meet the needs of more workers while at the 
same time assuring continued focus on meeting job preparatory and transfer demands by: 

• Finding new funding sources to increase the availability of evening and flexible 
enrollment courses at the colleges. 

• Continuing to invest in a technology infrastructure which will provide access in flexible 
formats for working adults. 

• Developing a mechanism for on-going “customer” feedback related to course variety, 
quality of instruction and extent to which incumbent workers are satisfied with college 
instruction. 

While colleges have been increasing their service to workers at the basic skills level, there is 
considerable opportunity to provide more service. The following strategies can address the 
needs of workers at the basic skills level. 

• Colleges can recruit from underrepresented industries: manufacturing; government; 
personal services; finance; construction; transportation; and utilities. 



• Since 27 percent of the basic skills upgrading students work for the larger employers, 
those employers could work with basic skills programs to develop work-based basic 
skills training at the employer work site. 

• Smaller employers and labor groups representing the agriculture, business, social and 
health services, and sales industries could form consortiums for the purposes of 
sponsoring industry-based basic skills training even though too few students from a 
single employer are enrolled to provide work-site training. 

Colleges could better meet needs of those seeking to enhance their work skills by: 

• Providing better advising for workers. 

• Gathering “customer feedback” related to course variety, quality of instruction and extent 
to which workers are satisfied with their college instruction. 

• Targeting workers in all industry areas, not just the service industries, which is the focus 
at present. 

• Increasing employer contract funded training. 
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EVALUATION ISSUES AND APPROACH 



This report meets the requirements of Senate Bill (SB) 5992 for “biennial program 
evaluations” by “operating agencies.” The Washington State Board for Community and 
Technical Colleges (SBCTC) conducted this research to provide data of use in directing the 
future of community and technical college training for employed workers. This report serves 
to focus attention both for those workers who enroll to upgrade skills at the college-level and 
those in pre-college adult basic education or English-as-a-Second Language programs. A 
more comprehensive study would have included data on basic skills worker training provided 
by community-based organizations. The data systems necessary for such analyses have just 
recently been developed. While future studies will be advantaged by inclusion of these 
programs, this report is necessarily limited to training within the community and technical 
college system. 

Several factors combine to make incumbent worker training the focus of this first biennial 
evaluation report: 

• Service to workers in college-level courses has declined throughout the 1990s. Part of that 
decline reflects the impact of the Baby Bust- the declining size of the population now in 
their 20s. Much of the decline, however, represents a shift of college resources to meet the 
demand for more job preparatory and transfer training. 

• The Workforce Training and Education Coordinating Board (WTECB) staff have reported 
an increased need for job skills enhancement training. A comprehensive understanding of 
the nature of upgrade training will facilitate the development of policy needed to best 
respond to that need. 

• The literature on basic skill training suggests that better retention of students results when 
basic skills training is offered in a specific context such as the family or work 
environment. While many basic skills students are not in the workforce, policymakers 
would be aided in directing resources to working students by knowing more about their 
employment status. 

“Job skills enhancement students” refers to employed adults enrolled in courses to upgrade 
their current job skills. Some workers enroll in college for the purpose of making a career 
change 2 . These job preparatory working adults are not included in this analysis. Their 
outcomes were examined by the WTECB staff as part of their review of programs under 
SB 5992. Even though some enrolled workers are excluded from this analysis because they 
are considered job preparatory, the term “worker” will be used in this report as synonomous 
with “job skills enhancement students.” 



2 While most students can be clearly identified as either job preparatory or upgrading, a gray 
area exists for some students. SBCTC arbitrarily categorizes students in the gray area as 
upgrading if they fail to declare a major and if their college codes them as other than job 
preparatory. 
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Workers typically enroll part-time and may take a single course rather than pursue a program 

of study. 

This report summarizes research conducted over the past five years. The wage and 

employment outcomes data was gathered in fall 1993 3 and analyzed this past spring. 

Data sources include: 

• In-class survey of 1,151 upgrading students at 8 representative community colleges, 
conducted in November 1990 (Nolte). 

• Focus group interviews with workers enrolled to upgrade job skills at six community 
colleges in spring 1990 (Borton). 

• Match between college records and the Unemployment Insurance system wage records 
for all 13,200 upgrading students who left the college during the 1991-92 academic year 
(Simmons) and of all respondents from the sample survey. 

• Enrollment records for all upgrading students from fall quarter 1990 to fall 1995. 



SUMMARY OF FINDINGS 



Trends in College Provided Job 
Skills Enhancement Training: 

In fall 1995, 26,721 workers enrolled 
to improve their job skills. They 
represented 1 2 percent of the total 
enrollment for fall, down from 16 
percent of total in fall 1990. Ten 
percent of that group was enrolled 
in basic skills instruction— literacy 
training for those with less than 
nineth-grade skill levels or English- 
as-a-Second Language instruction. 

The other 90 percent were enrolled in a wide variety of college level courses or a combination 
of developmental and college-level courses. 

Compared to 6 years ago, community colleges served 4,500 fewer workers in fall of 1995. 

All of that decline has been in college-level instruction. The basic skills effort for workers 
has grown about five percent a year over the same period. This decline is due almost entirely 
to factors impacting the colleges’ ability to meet the demands of the community. 



JobSkills Enhancement Enrollments 
Fall Quarter 




Community Colleges Technical Colleges 

q Workforce q Basic Skills q Workforce g Basic Skills 



3 These data were for workers leaving the colleges during the 1991-92 academic year. As this 
was the first year that technical colleges formed part of the new system, wage and 
employment data for those colleges are not included. 



Throughout the 1980s and 1990s, the demand for spaces at community and technical colleges 
has exceeded the resources needed to meet the demand. As a consequence colleges have had 
to make difficult choices, not offering some classes so that other courses meeting other needs 
could be offered. As a result of these choices, the opportunities for worker training have 
declined. 



There has been little change in 
demand for upgrade training. 

The pool of 20 to 39 years of age 
group most likely to enroll to 
upgrade job skills has stayed 
about the same size since 1990. 

While employment rates have 
varied for this group, the pool of 
employed 20- to 39-year-olds has 
also remained fairly constant. 

Thus the fall off in worker 
enrollment is not due to shifting 
population dynamics, but shifting 
priorities at the college. 

Colleges likely could have maintained worker enrollments at 30,000 (estimate for community 
and technical colleges combined in 1991) if funds to pay instructors’ salaries had not been 
needed to meet the demand for increased job preparatory training, transfer education, and 
basic skills training or to meet the needs of dislocated workers. 



Characteristics of 
Enrolled Workers: 

Workers who take classes 
at the college tend to be in 
their late 20s and early 30s. 
Most of those upgrading 
skills in college-level 
courses are women, but 
men are in the majority 
among workers in basic 
skills classes. Being older, 
more than a third of these 
students were parents with 
dependent children in 
addition to being workers 
and students. Those taking 
college-level classes were a 
well-educated group with 



Enrolled Worker Characteristics 



Median Age 
Age 25 and Older 

Women 
Of Color 

Single Parents 
Couple with Children 

Previously Took Some College 
AA Degree or Higher 



College-Level 


Basic 

Skills 


31 


28 


68% 


65% 


56% 


43% 


11% 


44% 


8% 


9% 


29% 


44% 


65% 


15% 


24% 


0% 
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Washington State Population, 20-39 Year Olds 




65 percent having been to college. Some 15 percent of basic skills students also had been to 
college, including those who attended college in another country before immigrating to the 
United States. 



In fall 1995, most job skills 
enhancement basic skills 


Worker Enrollment Pattern 




students enrolled for their 
first time or had started 
college just recently. 




College-Level 


Basic 

Skills 


Those taking college-level 
courses, on the other hand, 


Time of First Enrollment 
Last 2 Years 


40% 


63% 


exhibited an ongoing 


3-4 Years 


14% 


12% 


relationship with the 


5-9 Years 


20% 


11% 


college. The majority had 


10 or More Years 


16% 


4% 


taken classes some time 
ago in addition to the class 


Enrolled Part-Time 


75% 


72% 


or classes taken in fall 
1995. Some 16 percent had 


Enrolled at Night 


54% 


52% 


taken classes some 1 0 or 


Enrolled off Campus 


20% 


25% 


more years ago. Workers 


Took Vocational Courses 


59% 




appear to be lifelong 


Took Academic Courses 


49% 




learners who return to the 


Took Below College Courses 


6% 


100% 


college periodically to 
upgrade skills or meet 


Employer Contracted Courses 


11% 


NA 


personal interests. 


State or Student Funded Courses 


89% 


NA 



Just over half the workers enrolled at night. Nearly half took at least one academic course, 
though the majority took a vocational class. About six percent of college-level workers took 
at least one developmental studies class (below college-level) along with college-level 
classes. 

While 59 percent of all students enrolled in some vocational classes, the FTE generated in 
academic classes— taken alone or along with vocational classes— was substantial. Workers 
operate in all kinds of contexts and, thus, need all kinds of upgrading. Some workers, for 
example, teach music for a living, so it is not surprising that part of the humanities FTE 
included music performance classes such as community band or choir (typically offered to 
the community on a self-supporting basis). Spanish language skills are needed by many 
workers, and, thus, foreign language courses, mostly Spanish, comprise four percent of the 
FTE taken by these workers in fall 1995. Many workers seek to better understand the social 
and personal forces that impact the work world and, thus, enroll in psychology, sociology, 
and political science classes. Some workers need to enhance their understanding of 
chemistry, the biological sciences, and math; thus, each area represents four percent of 
the FTE. 
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Job Skills Enhancement FTE by Course Type 
(College-Level Courses Only) 



All Other 
14% 



Other Prof/Tech 
6% 



Professional/ 

Technical 



Secretarial 

5% 



Business/ 

Service 
Occupations 

Service Occupations I 
5% 

Business Management 
6% 



Natural 



Sciences 



Chemistry, Other Sciences 
4% 

Biological Sciences 




Computer Related 
8 % 

Accounting 
4% 



Other Humanities 
7% 



English /Speech 
7% 



Humanities 



Welding/Sheet Metal, Repairs 
5% 



Foreign Language 
4% 

Soc/ Anthropology 
3% 

Psychology Social 

Construction Trades Science 

5% History /Political Science 

Mechanics, 4% 

Repair 



Fall 1995 - All funds - excludes ABE/ESL and Developmental 



Among occupationally specific courses, the largest single area of study was in computer- 
related courses at eight percent of the FTE. The secretarial; business management; various 
professional/ technical areas, including health fields; and construction trades each accounted 
for five or six percent of the FTE. It is clear then that no one area of study dominates the 
interest of job skills enhancement students. They are enrolled in all areas of the curriculum. 

The median earnings of those who enroll for upgrade training was $10.81 for those in 
college-level programs and $7.93 for those in basic skills. Most worked in smaller companies 
with more than one-third working in firms of fewer than 50 employees. Some 32 percent of 
college-level workers were from firms with 500 or more workers, compared to 27 percent of 
the upgrading basic skills students. 

The following hypothetical student profiles based on the focus group interviews (Borton) and 
college enrollment records help clarify the diversity among workers enrolled at the college. 
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Marian, a 28-year-old white female, has been working as a full-time personnel assistant with 
a mid-sized family clinic for the past five years. Before enrolling at her current college in fall 
1994, Marian had already taken some courses at a business college. In fall 1994, she was 
enrolled in an evening five-credit, state-supported vocational course in personnel 
management. Marian did not receive any need-based financial aid nor was her tuition 
reimbursed by her company. 

Marian is uncertain whether she will continue at the college after fall term. She worries about 
meeting her family obligations, working, and being a student. She has limited time to study at 
night or on the weekends. Assignments that require library work are especially difficult for 
her. Despite being successful at the business college she attended, she is now concerned 
about failing her course because of the number of years since she was last in school. Marian 
expects that her current employer will reward her training effort with a salary increase. 



Rex, a 35-year-old white male, works full-time for an electronics manufacturing firm that 
employs about 170 people. He is taking a night-time vocational course to upgrade skills at his 
current job. After completing the course, he does not plan to return to the college for several 
years. He figures that his current course work will increase his productivity, a high priority 
for his current employer. Rex’s employer is paying for his course, provided he achieves at 
least a C grade. 

If Rex could change one thing about taking courses at the college, he would change the time 
and place of his course. Since the course will benefit his company as much as himself, he 
thinks the course should be offered during work hours at his job site. 



Craig works full-time at a small restaurant with fewer than 20 employees. He has been 
working there for two years. He likes the contact with customers at his current job but is 
unhappy with the evening and weekend work schedule. Craig is single, white, and in his late 
twenties. Craig started taking basic skills classes in fall 1990. He plans to take courses again 
at the college in winter and spring quarters and to pass his GED. Craig expects that this 
course may help in seeking a higher paying job. Even more importantly, he expects a new job 
will allow him more leisure time than his current employment. Craig paid the cost of 
attending himself without aid from the college or his company. 



The workers who enroll 
in college for their 
upgrade training 
represent a different 
profile than the typical 
worker by industry. 
Workers in basic skills 
programs differ from 
those in college-level 
training. The job skills 
enhancement basic 
skills students were 
over-represented in 
agriculture, business, 
health and social 
services, and sales. 
College-level students 
were over-represented 
in only one area- 
business, health and 
social services. 



% by Industry — Workers in Training vs. 
All Washington Workers 



Manufacturing 



Trade (Wholesale 
& Retail) 




Agricultural 

Transportation, 
Public Utilities U 



Finance, Insurance, 
Real Estate 

Personal Services 
Government 



g Basic Skills 

□ College-Level 

□ All Washington Workers 






Business, Health, 

Social Services i; : - jy 



5% 10% 15% 20% 25% 30% 



Clarifying Assumptions About Job Skills Enhancement Worker Training : Some of the 
widely held beliefs about worker training at community and technical colleges are not 
substantiated by the data. Specifically, the following conclusions stem from the data and 
contradict common assumptions. 



Common Assumptions 


What the Research Tells Us 


Employers sponsor most worker 
course taking. 


Upgrading students enroll primarily without the benefit 
of employers sponsorship either through course 
contracting or through tuition reimbursement. Just 
27 percent of workers receive tuition support from their 
employer. Another 1 1 percent enroll in employer 
contracted courses. 
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Common Assumptions 


What the Research Tells Us 


Working adults enroll in night time 
vocational courses 


Upgrading students enroll during the day as well as at 
night and in academic courses as well as in courses 
specifically designed for upgrading in specific 
occupations. Some 46 percent enroll during the day, 
and 49 percent take at least one academic course. 


Working adults get their training from 
a single provider— in this case, the 
college. 


Workers use colleges as just one of several sources of 
continual training. For the majority, the college courses 
are not their only source of training. Two-thirds of 
workers had also participated in training other than that 
offered by the college including courses at other 
colleges, privately offered training, and employer 
training. 


The beginning and end of upgrade 
training can be identified for use in 
performance assessment. 


Upgrade training is undertaken as a continual process. 
The notion of “starting” training and “completion” does 
not apply. The students return to college on an as 
needed basis. Evaluation based on an assumption that a 
point in time can be defined as the pre-training or post- 
training period does not correspond to the actual 
behavior of workers. 



Outcomes Assessment: Customer satisfaction surveys are an appropriate component in any 
performance assessment system, but they are especially central to assessment of college 
services to workers. Working adults report that they expect the benefits of training to be in 
areas best assessed by use of a satisfaction survey. The top benefits of training listed by 
workers were: 



Personal satisfaction 


79% 


Increased job responsibility 
Increased salary 
Increased job security 
Improved quality of life 


69% 

68% 

63% 

61% 



Job skills enhancement students 
reported a high level of 
satisfaction with the training 
provided at the colleges. Some 
90 percent would recommend 
the college to a friend or co- 
worker. The satisfaction ratings 
were highest for the quality of 
instruction and the access to up- 
to-date equipment and lowest 
for availability of courses and 
help with selecting courses. 

Given the cost of post-training 
survey work, SBCTC has not 
conducted a study that could 
reveal valuable information about the extent to which training led to the expected outcomes 
of increased job responsibility, job security, and improved quality of life as well. Such data 
collection is needed periodically. 

SBCTC staff undertook this analysis with the goal of reporting post-training earnings as one 
of the outcomes measures for job skills enhancement training. However, an analysis of 
factors which contribute toward explaining differential earnings of former students found the 
extent of upgrade training to be an unimportant factor. Key factors impacting post-training 
wage levels, in order of importance, are age, gender and education level prior to training (see 
Appendix B). The extent of this relationship can be seen by comparing wages increase with 
prior education levels for college-level job skills enhancement students. 
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Post Training Wage Group by Prior Education of Incumbent Workers (College-Level) 



Wage 

Group 


Below 

HS 


GED 


High 

School 


Post 

HS+ 


Certificate 


AA 


BA 


$15/ Hour+ 


11% 


17% 


16% 


28% 


30% 


38% 


49% 


$10-14.99 


24% 


33% 


24% 


33% 


38% 


33% 


33% 


$6.50-9.99 


27% 


33% 


35% 


28% 


23% 


22% 


16% 


$6.49/Less 


38% 


16% 


24% 


11% 


9% 


6% 


2% 



This finding implies that the value-added by college attendance cannot be measured by post- 
training earnings. Given the finding that such earnings relate not to training but to 
“environmental” factors, wage data reported here are not regarded as an outcome of the 
training. 
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It is possible that a pre/post wage comparison could yield a differential wage gain for those 
with more hours in upgrade training. This is not a strong possibility, however, given the 
strength of the association between post-training wage and age and education level at the 
time training “started.” 



POLICY IMPLICATIONS 

Colleges provide one of many sources of quality upgrade training for workers and should 
continue to be funded to provide such training. These life long learners rate their college 
experience positively. 

The outcomes indicator ranked by workers as most important was personal satisfaction. Such 
an outcome can best be measured by consumer satisfaction surveys. A system for worker 
surveys needs to be developed. Other than the in-class survey described here, no systematic 
effort has been made by SBCTC or WTECB to conduct direct customer satisfaction surveys 
related to the incumbent worker training mission. WTECB has conducted employer 
satisfaction surveys related to those workers who participate in employer-contracted courses. 
Given the life-long learning approach of workers, they could be surveyed immediately after a 
course ends. There is no need to wait to learn if the student has “left” the college in the next 
term as is the protocol for job preparatory surveys. 

While colleges have been increasing their service to workers at the basic skills level, there is 
considerable opportunity to increase that level of service. Most of the job skill enhancement 
basic skills students (73 percent) work for smaller firms of fewer than 500 employees. In fact, 
38 percent work in firms with fewer than 50 employees. Colleges could work with employers 
and labor groups to form a consortium representing the agriculture, business, social and 
health services, and sales industries for the purposes of sponsoring industry-based basic skills 
training. Since there are too few students from a single employer enrolled to provide work- 
site training, these consortia could provide training in an industry environment instead. For 
the 27 percent of students working in large companies with 500 or more employees, the 
college and employers could work together to provide work-based basic skills training at the 
employer work site. Basic skills outreach could also be extended to industries other than 
agriculture, business, social and health services, and sales, which are now over represented in 
the basic skills enrollment. 

New funding sources may be needed if colleges are to continue to meet job preparatory and 
transfer demands while also effectively meet the need to serve more, not fewer workers. 

With additional funding colleges could increase the availability of evening and flexible 
enrollment courses. The state’s continued investment in an electronic infrastructure, which 
will allow enrollment in more flexible formats for working adults is also critical. 

Colleges could better meet worker needs by providing better advising and targeting workers 
in all industry areas, not just the service industries as is the focus at present. Just as the state 
needs to develop a systematic way of gathering “customer feedback” related to job skills 
enhancement 



training, colleges could improve their programs by doing so also. The consumer surveys 
should focus on the issues of importance to students: course variety; quality of instruction; 
and extent to which workers are meeting their needs via college instruction. 

While administrative data linking is an inexpensive way to gather outcomes data, post- 
training data are not shown to be useful for evaluating this mission of the college. 
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Appendix A: 

COMMUNITY COLLEGE JOB UPGRADING/ 
RETRAINING SURVEY, NOVEMBER 1990 



As part of his dissertation study, Walt Nolte designed and administered The Community 
College Job Upgrading/Retraining Survey in November of 1990. The survey results, along 
with demographic information from the SBCTC Student Management Information System, 
provided the basis for the profile of upgrading and career students and the student evaluation 
of the expected outcomes and satisfaction with community college education. 

Questionnaire 

The eight page questionnaire was designed based on previous SBCTC survey instruments, a 
literature review, and results from the Borton study. The questionnaire was pilot tested with a 
group of employed students at Tacoma Community College. The final instruction took about 
15 minutes to complete and was administered during the class session by staff hired 
specifically for that purpose. 



A copy of the questionnaire is attached. The main areas covered in the instrument were: 



-Reasons for enrolling 
-Reasons students enrolled at 
a community college 
-Expected Outcomes from course 
-Barriers to enrolling 
-Measures of student satisfaction 



-Comparisons with other training experienced 
-Future needs related to training 
-Current job status 
-Tuition reimbursement status 
-Relationship of salary and work 



No background questions were asked because it was expected that such information could be 
obtained from matching survey responses with data collected at registration. Students were 
advised, both in writing and orally, of this use of the social security number they provided on 
the questionnaire at the time of the administration of the survey. 

Survey Sample Frame 

The sample for this survey was drawn from all students enrolled in courses identified as 
serving employed students. From six to twelve classes were identified for the sample by 
administrators at eight community colleges: 

Big Bend Community College 
Edmonds Community College 
Highline Community College 
North Seattle Community College 



Skagit Valley Community College 
South Puget Sound Community College 
Spokane Community College 
Tacoma Community College 
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Each college has a history of providing educational services to employed students and a 
reputation for providing services to private sector business and industry. These colleges 
represent the mix of all community colleges in terms of urban, suburban, and rural settings; 
size; and location. 

Each college was asked to survey 125 students, but due to differences in the size of colleges, 
the responses by college generally exceeded this norm. Administrators were aided in the 
course selection by a computer program that pulled student characteristics information and 
displayed the courses most likely to contain upgrading and retraining students. The program 
also provided information on the mix of such courses in terms of day versus evening and 
vocational, academic, or basic skills mix. 

Administration 

Staff at each of the eight colleges administered the surveys to 1,151 students during the sixth 
through the eighth week of fall quarter 1990. Campus staff followed a common procedure to 
contact the faculty and gain agreement to use class time for the survey. A single individual 
administered all surveys and used a prepared script to announce the survey process and 
answer common questions. Only those students who said they were employed and enrolled to 
improve their job skills or prepare for a career change were asked to complete the 
questionnaire. 

Students who wanted a copy of the survey results were asked to stop by the dean’s office in 
June of the coming year to obtain copies. 

SBCTC staff completed data entry of all surveys using a locally designed data entry program, 
which disallows responses outside the range of those allowed on the questionnaire. 

Additional Data 

In addition to survey results, the analysis of upgrading and retraining students was partly 
based on data from two other sources: 

• Information provided by survey respondents at the time of registration. This information 
included student purpose for attending, planned length of enrollment, current job status, 
race, gender, and types of courses in which the student enrolled. 

• Information on student job status as of four months earlier related to an estimated annual 
salary and hours of work based on a match of social security numbers with the 
unemployment insurance data files maintained by the Washington State Employment 
Security Department. 

Data from both sources were matched to the survey responses using the social security 
numbers provided by survey respondents. The survey form included an explanation of the use 
that would be made of the social security number if provided. Of the 1,151 students who 
completed the survey, 109 (9.5) percent left the social security number blank. Additionally, 



84 provided numbers that did not match with the registration data. Thus the above data were 
not available for 17 percent of the survey respondents. 



DEFINING THE DEGREE OF CERTAINTY 
IN USING THESE SURVEY FINDINGS 

Factors Influencing Certainty of Findings 

All surveys have some degree of uncertainty— that is the concern that findings do not 
represent the beliefs attitudes, opinions or behaviors of the population studied. If uncertainty 
is low, the findings are far more useful. For many, the first and only factor that comes to 
mind when considering the degree of certainty attached to findings from a survey is sampling 
error. Sampling error refers to the degree to which the sample represents the total group or 
population for which information was sought. But there are three other factors that are 
important in determining the degree of certainty of survey findings: 

• Measurement Error: The degree to which the questions asked truly 

measures what the researcher intended. 

• Non-Coverage: The degree to which the entire population had 

an opportunity to be included in the sample. 

• Non-Response Bias: Bias created if the answers from non- 

respondents were likely to differ considerably 
from respondents and the number of non- 
respondents was large enough to impact the 
findings. 

Researchers attempt to control the four factors mentioned above to assure the highest level of 
certainty possible given the resources available to conduct the survey. 

In the case of this study, findings from The Job Upgrading/Retraining Survey are regarded as 
fairly high in certainty because the researchers were able to minimize bias and errorpart. 
Nevertheless, some uncertainty remains due primarily to sampling error and non-coverage. 

Measurement Error 



The following factors helped reduce uncertainty regarding measurement: 




• Nolte had extensively pilot-tested the questionnaire with students at his community 
college. 

• The questionnaire contained mostly standard questions which had been previously 
pilot-tested and used in a variety of other settings. 

• Some questions were asked in several ways. 

• The survey was short and well structured. 
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5 ? 



Respondents did not have to work hard. Their interest was maintained as they were 
led through the survey. 






For some questions, respondents were asked to indicate if each response did or did not apply 
to them. Often a response was left blank. Analysis of this survey assumed a blank response to 
mean “does not apply.” (If this assumption is false, the level of measurement error could be 
considerable.) 

Sampling Error 

Courses were used as a basis for sampling students. How well these students represented all 
upgrading and retraining students depended on the courses selected. Deans of instruction or 
vocational deans at each campus selected courses, which they regarded as representative of 
courses where upgrading and retraining students were most likely to enroll. Course profile 
data was provided to assist the deans in their selection. While this selection process was not 
as free from error as random selection, it is assumed that the judgments made resulted in 
minimal error. 

The course-based cluster sampling technique does not allow for exact specification of 
sampling of error. Based on sample size and assumed representative courses, the sampling 
error is assumed to be not greater than plus or minus five percent. 

Non-Coverage 

Not all upgrading and retraining students had an opportunity to be in the survey sample and 
thus there is uncertainty in the findings due to non-coverage. Specifically, those that did not 
have an opportunity to be in the sample included: 

• Any student in a selected course when the faculty could not afford 15 minutes of class 
time to administer the survey. 

• Upgrading and retraining students in the following courses: 

- English-as-a-Second Language (ESL) 

- Adult Basic Education (ABE) 

These students were excluded because of the reading skill required to complete the 
survey. Students in developmental studies courses were included, however, and the 
results can be said to fairly represent upgrading and retraining students above the 
literacy training level. 

• Upgrading and retraining students in colleges not included in the study. It was 
assumed that the participating colleges adequately represented the community college 
system. 

For some analysis, findings were based on a combination of survey data and registration data. 
SBCTC matched the two sources of data based on social security numbers supplied by 
respondents. Not all students supplied their social security numbers. Thus findings based on 



registration and survey data can be regarded as high in certainty only for those who supplied 
their social security numbers. 

Non-Response Bias 

In this study, the non-response rate is unknown as the number of upgrading and retraining 
students in each course was unknown. Students in the course were asked orally and at the 
start of the survey to determine if they qualified to complete the survey. Only those who said 
they were currently employed and enrolled to improve their job skills or prepare for a career 
change were asked to complete the survey. It is possible that some who qualified did not 
respond. It is also possible that some non-qualifying students completed the survey. 

Non-response may have also resulted from upgrading and retraining students not being in 
class on the day of the survey. Given the timing of the survey early in the quarter and the ease 
of completing the instrument, it is likely that most upgrading and retraining students were 
present and most responded to the survey. Thus, this in-class survey like most of that type, 
probably had considerably less non-response bias than mail or phone surveys of similar 
populations. 

Conclusion 

The findings of this survey can be thought of as fairly accurate in representing the opinions, 
beliefs, attitudes, and behaviors of upgrading and retraining students above the literacy 
training level in Washington community colleges. 
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JOB UPGRADING/LRETRAINING SURVEY 



Are you currently employed? Are you enrolled in college to improve your job skills or prepare for a 
career change? If your answer to both questions was YES, we would appreciate your help by 
completing this survey about students enrolled ot improve job skills. 

Your answers are a valuable source of information to help Washington community colleges improve 
instruction and support services forworking students. Your social security number is needed to match 
to the demographic information.you have already provided the colleges. Your responses will remain 
confidential. 

Name (optional): 

Social Security Number: - - 



Q-l . How would you describe your course(s) in relation to your current job? (Circle the number 
that best applies to you— one only.) 



1 . STRONGLY RELATED TO JOB 

2. SOMEWHAT RELATED TO JOB 

3. NOT RELATED TO JOB 

4. UNCERTAIN 



Q-2. Which of the following apply to you as a reason for enrolling at your community college? 



Applies 



1 TO TAKE COURSES RELATED TO CURRENT JOB 1 

2 TO PREPARE FOR A NEW JOB AT EXISTING PLACE 

OF EMPLOYMENT 1 

3 TO PREPARE FOR A NEW JOB AT ANOTHER PLACE 

OF EMPLOYMENT 1 

4 TO PREPARE FOR A CAREER CHANGE 1 

5 TO EXPLORE A NEW CAREER DIRECTION 1 

6 TO PREPARE FO INCREASE IN JOB COMPLEXITY 1 

7 TO MOVE FROM TEMPORARY WORK INTO A 

PERMANENT CAREER 1 

8 OTHER 

(Please Specify) 



Does Not 
Apply 
2 

2 

2 

2 

2 

2 

2 



Q-3. Are there changes in your place of employment requiring you to upgrade or retrain? 



Applies Does Not 



Apply 

1 NO CHANGES 1 2 

2 COMPANY IS BEING EXPANDED 1 2 

3 COMPANY IS BEING DOWNSIZED OR LIQUIDATED 1 2 

4 COMPANY HAS BEEN ACQUIRED OR MERGED 1 2 

5 CHANGE IN MAJOR CUSTOMERS 1 2 

6 CHANGE IN MAJOR PRODUCTS OR SERVICES 1 2 

7 CHANGE IN COMPANY OWNERSHIP OR MANAGEMENT 1 2 
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Applies 



Does Not 



Apply 

8 INCREASED COMPETITION FORM OTHER FIRMS 1 2 

9 CHANGES IN COMPANY TECHNOLOGY 1 2 

10 CHANGES IN COMPANY LOCATION 1 2 

1 1 COMPANY IS BEING CLOSED AND NO OTHERS 

REQUIRE MY SKILLS 1 2 

12 COMPANY’S MAJOR CUSTOMER(S) REQUIRES 

TRAINING AT OUR FIRM 1 2 

13 OTHER 

(Please Specify) 



Q-4. Why did you decide to go to this community college? 



Applies 



1 TYPE OF COURSES AND PROGRAMS 1 

2 RECOMMENDED BY EMPLOYER 1 

3 RECOMMENDED BY CO-WORKERS 1 

4 CONVENIENT LOCATION 1 

5 CONVENIENT COURSE TIME 1 

6 AFFORDABILITY 1 

7 OTHER 



(Please Specify) 



Does Not 
Apply 
2 
2 
2 
2 
2 
2 



Q-5. What are your future expectations regarding your current or planned course(s)? 

Applies Does Not 



. . ■ - Apply 

1 INCREASE JOB SECURITY 1 2 

2 INCREASED JOB RESPONSIBILITY 1 2 

3 INCREASED SALARY 1 2 

4 NEW POSITION AT EXISTING PLACE OF EMPLOYMENT 1 2 

5 NEW POSITION WITH ANOTHER EMPLOYER 1 2 

6 START OWN BUSINESS OR PRACTICE 1 2 

7 IMPROVED QUALITY OF LIFE 1 2 

8 PERSONAL SATISFACTION 1 2 

9 BETTER LIFE FOR CHILDREN 1 2 

10 OTHER 

(Please Specify) 



Q-6. Based on what you have learned in your classes so far, do you think your community college 
experience will help you meet these expectations? 



Applies 



1 YES, IMMEDIATE BENEFIT 1 

2 YES, LONG TERM BENEFIT 1 

3 INDIRECT BENEFIT 1 

4 NO BENEFIT 1 

5 UNSURE 1 



Does Not 
Apply 
2 
2 
2 
2 
2 
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Q-7. How much impact do you expect your community college experience to have on your 
wages? 



Applies 



1 LARGE INCREASE 1 

2 SMALL INCREASE 1 

3 NO INCREASE 1 

4 DECLINE 1 

5 UNCERTAIN 1 



Does Not 
Apply 
2 
2 
2 
2 
2 



Q-8. Did you have to overcome any of the following to enroll in community college? 



Applies Does Not 



INSTITUTIONAL BARRIERS Apply 

1 DIFFICULTY IN SCHEDULING COURSES AND WORK 1 2 

2 EMPLOYER’S TRAINING AND EDUCATION POLICIES 1 2 

3 INCONVENIENT LOCATION AND COURSES 1 2 

4 LACK OF RELEVANCY OF COURSES 1 2 

5 INCONVENIENT LENGTH OF COURSES 

(WEEKS, MONTHS, QUARTERS, ETC.) 1 2 

6 INCONVENIENT TIME OR DAY OF COURSES 1 2 

7 LACK OF COLLEGE SUPPORT SERVICES 

(ADVISING, COUNSELING, ETC.) 1 2 

8 DIFFICULT REGISTRATION PROCESS 1 2 

9 LACK OF KNOWLEDGE ABOUT COURSES AND PROGRAMS 1 2 

10 COLLEGE PLACEMENT TESTS 1 2 

11 COURSE PREREQUISITES 1 2 

12 OTHER 

(Please Specify) 



Which barrier was the greates problem for you? 

(Number from above) 



Q-9. Did you have to overcome any of the following in order to attend community college? 



Applies Does Not 

Apply 

1 LACK OF CONFIDENCE 1 2 

2 FEAR OF FAILURE 1 2 

3 FEELING OF BEING TOO OLD 1 2 

4 LACK OF PERSONAL INTEREST 1 2 

5 FAMILY RESPONSIBILITIES 1 2 

6 LACK OF SPOUSE OR FAMILY SUPPORT 1 2 

7 FEAR OF TRYING SOMETHING NEW 1 2 

8 PHYSICAL DISABILITIES 1 2 

9 CHILDCARE 1 2 

10 FINANCES..: 1 2 

11 TRANSPORTATION 1 2 

12 OTHER 

(Please Specify) 



Which barrier was the greates problem for you? 

(Number from above) 
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Q-10. In general, how satisfied are you with your community college with regard to eachof the 
following areas? 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 



Very 

Unsatisfied 



QUALITY OF INSTRUCTION 1 

HELP WITH SELECTING COURSES 1 

INFORMATION ON CONDUCTING A 

JOB SEARCH 1 

VARIETY OF COURSES OFFERED 1 

USEFULNESS OR RELEVANCY OF 

TRAINIG TO CURRENT JOB 1 

AVAILABILITY OF CLASSES AT 

THE TIMES I COULD ATTEND 1 

INTERACTION WITH FACULTY 

OUTSIDE OF THE CLASSROOM 1 

ACCESS TO UP-TO-DATE 

EQUIPMENT 1 

SERVICES FOR DISABLED 1 

TESTING OR ASSESSMENT 

SERVICES 1 

FINANCIAL AID 1 



Somewhat 


Somewhat 


Very 


Does Not 


Unsatisfied 


Satisfied 


Satisfied 


Apply 


2 


3 


4 


5 


2 


3 


4 


5 


2 


3 


4 


5 


2 








2 


3 


4 


5 


2 


3 


4 


5 


2 


3 


4 


5 


2 


3 


4 


5 


2 


3 


4 


5 


2 


3 


4 


5 


2 


3 


4 


5 



Q- 1 1 . Based on what have learned in your course(s), do you expect your community college 
experience to increase your on-the-job productivity? 

1 YES (go to Q-12) 

2 NO (skip to Q-1 3, next page) 

3 UNSURE (skip to Q-1 3, next page 

Q-12. Will your work productivity be increased in the following year? 

Applies Does Not 



Apply 

1 ABILITY TO WORK “SMARTER NO HARDER” 1 2 

2 ABILITY TO WORK WITHOUT ERRORS 1 2 

3 ABILITY TO WORK BETTER WITH CO-WORKERS 1 2 

4 BETTER KNOWLEDGE OF JOB RESPONSIBILITIES 1 2 

5 ABILITY TO WORK AS A PART OF A TEAM 1 2 

6 ABILITY TO WORK WITH LESS SUPERVISION 1 2 

7 ABILITY TO SUPERVISE OTHERS BETTER 1 2 

8 SHORTER TURN-AROUND TIME OF PRODUCT OR SERVICE 1 2 

9 ABILITY TO BE CREATIVE, FLEXIBLE, OR PROBLEM SOLVE 1 2 

10 ABILITY TO READ AND WRITE EFFECTIVELY 1 2 

1 1 ABILITY TO LEARN NEW SYSTEMS OR PROCEDURES 1 2 

1 2 ABILITY TO MAKE PRESENTATIONS 1 2 

1 3 ABILITY TO USE MATH AT WORK 1 2 

14 ABILITY TO MAKE DECISIONS INDEPENDENTLY 1 2 

13 OTHER 

(Please Specify) 



Q-13. Other than high school, have you been involved in other training and education programs? 

1 NO (skip to Q-1 5) 

2 YES 

Q-1 4. If you have been involved in other training and education programs, how would you 

compare your experience to the community college? (Select 4 if you had not participated or 
cannot compare the item.) 



Worse 



1 EMPLOYER SPONSORED TRAINING 1 

2 MILITARY 1 

3 PRIVATELY SPONSORED SEMINARS 

AND WORKSHOPS 1 

4 FORMAL EDUCATION 1 

5 OTHER ■ 

(Please Specify) 



About Does Not 

Same Better Apply 
2 3 4 

2 3 4 

2 3 4 

2 3 4 



Q-1 5. Do you see yourself as continually needing training in the future? (Circle the number that 
best applies to you— one only.) 



1 YES, EVERY YEAR 

2 YES, EVERY COUPLE OF YEARS 

3 NO 

4 UNCERTAIN 



Q-1 6. Do you expect to attend to a four-year institution in the next five years? (Circle the number 
that best applies to you— one only.) 



1 NO (skip to Q-1 8, next page) 

2 NO, ALREADY HAVE A FOUR YEAR DEGREE (skip to Q-1 8, next page) 

3 YES, TO WORK TOWARDS A FOUR YEAR DEGREE 

4 YES, FOR CONTINUING EDUCATION 

5 UNSURE 



Q- 1 7. If you expect to transfer, what institution do you plan to attend? (Circle the number of the 
institution that you are most likely to attend— one only.) 



1 THE EVERGREEN STATE COLLEGE 

2 WESTERN WASHINGTON UNIVERSITY 

3 CENTRAL WASHINGTON UNIVERSITY 

4 EASTERN WASHINGTON UNIVERSITY 

5 WASHINGTON STATE, MAIN CAMPUS 
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6 WASHINGTON STATE, TRI-CITIES 

7 WASHINGTON STATE, VANCOUVER BRANCH 

8 UNIVERSITY OF WASHINGTON, MAIN CAMPUS 

9 UNIVERSITY OF WASHINGTON, TACOMA BRANCH 

10 UNIVERSITY OF WASHINGTON, BOTHELL/WOODINVILLE BRANCH 

1 1 PRIVATE COLLEGE IN WASHINGTON 

12 OUT-OF-STATE 

Q- 1 8. What if your approximate start date at the 4-year institution? (Select one only.) 

1 ALREADY ENROLLED 

2 WINTER OR SPRING 1991 

3 SUMMER OR FALL 1991 

4 WINTER OR SPRING 1 992 

5 1992-93 ACADEMIC YEAR 

6 1993-94 ACADEMIC YEAR 

7 1994-95 ACADEMIC YEAR 

8 NOT SURE 

Q-19. Would you recommend the community college to your friends, co-workers, or 
acquaintances? 

1 YES 

2 NO 

3 UNCERTAIN 

Q-20. If you could go back, knowing what you now know, would you still attend this commlunity 
college? 

1 NO 

2 YES 

3 UNCERTAIN 

Q-21. What is the title of your current job (also describe duties)? 

TITLE 

DUTIES 



Q-22. How would you describe the level of your joib in relationship to your skills? (Circle the 
number that best applies to you— one only.) 

1 JOB IS DEFINITELY BENEATH MY SKILL LEVEL 

2 JOB IS SOMEWHAT BENEATH MY SKILL LEVEL 

3 JOB IS APPROPRIATE FOR MY SKILL LEVEL 

4 JOB IS TOO ADVANCED FOR MY SKILL LEVEL 

5 DON’T KNOW 

Q-23. How many people are employed where you work? 

1 . FEWER THAN 20 STAFF 

2. 20-49 STAFF 

3. 50-249 STAFF 

4. 250 OR MORE STAFF 
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Q-24. How long have you been employed at your current place of work? (Circle the number that 
best applies to you-one only.) 

1 LESS THAN ONE YEAR 

2 ONE TO THREE YEARS 

3 FOUR TO FIVE YEARS 

4 MORE THAN FIVE YEARS 

5 DON’T KNOW 

Q-25. What is the nature of the business of your current employer? (Circle the number that best 
applies to you-one only.) 

1 AGRICULTURE, FORESTRY OR FISHING 

2 MINING 

3 CONSTRUCTION 

4 MANUFACTURING 

5 TRANSPORTATION, COMMUNICATION, ELECTRIC, GAS, OR SANITARY SERVICES 

6 WHOLESALE TRADE 

7 RETAIL TRADE 

8 BUSINESS, LEGAL, OR SOCIAL SERVICES 

9 HEALTH OR EDUCATIONAL SERVICES 

10 FINANCE, INSURANCE, OR REAL ESTATE 

11 CONSUMER SERVICES 

12 PUBLIC ADMINISTRATION 

13 OTHER 

(Please specify) 



Q-26. Who paid the tuition or fees for the course(s) in which you are currently enrolled? (Circle the 
number that best applies to you-one only.) 

1 100% EMPLOYER PAID 

2 PARTIAL EMPLOYER PAID, PARTIAL SELF-PAID 

3 100% SELF-PAID 

4 COLLEGE FINANCIAL PROGRAM 

5 SELF-PAID AND COLLEGE FINANCIAL PROGRAM 

6 A COMBINATION OF SELF, EMPLOYER, AND FINANCIAL AID 

7 OTHER ; 

(Please specify) 



Q-27. Did you receive financial assistance from any source for textbooks, supplies, or other 
educational costs? 



1 YES 

2 NO 

3 UNCERTAIN 
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Q-28. How would you describe your current salary in relationship to your work skills? 
(Circle the number that best applies to you-one only.) 

1 SALARY IS DEFINITELY BENEATH MY SKILL LEVEL 

2 SALARY IS SOMEWHAT BENEATH MY SKILL LEVEL 

3 SALARY IS APPROPRIATE FOR MY SKILL LEVEL 

4 SALARY IS SLIGHTLY MORE THAN MY SKILL LEVEL 

5 SALARY IS DEFINITELY HIGHER THAN MY SKILL LEVEL 

6 DON’T KNOW 

Q-29. If your company has a tuition reimbursement program, please describe: 



Q-30. Please use the space below to discuss the strengths or weaknesses of the community college 
programs and services - 



Thank you for your assistance. 
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Appendix B: 

STATISTICAL ANALYSES OF IMPACTS 
OF POST-PROGRAM WAGE 

Results of Discriminant Function Analyses of Wage Group Criterion 



A. Accuracy of Predictions 


87% 


B. Specific Variables Predicting the Wage Group 




Criterion 


Age 


.73 


Gender 


.48 


Previous Education Level 


.39 


Race 


.05 


Credit Level Earned 


-.03 


Wilks Lambda 


.87 



Hourly wage groups measured at 4 levels: (l=low - 4=high) 



Codes for Variables: 

Race: 1 = all non-white ethnicities; 2 = white 
Gender: 1 = female; 2 = male 

Age: 1 = under 25; 2 = 25-39; 3 = 40-54; 4 = 55 and over 
Stucrlvl: 1 = 45 credits or less; 2 = more than 45 credits 

Edu: 1 = non-h.s. & GED; 2 = h.s. diploma; 3 = some post high school & certificates; 
4 = Associate’s degree; 5 = Bachelors or more 
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Appendix C: 

FOCUS GROUP INTERVIEWS 



As part of her dissertation study, Nancy Borton designed and implemented focus group 
interviews used as a data source in this study. She used an ethnography approach, which 
seeks to make sense of “the whole” by studying the meanings people and organizations 
assign phenomena in a context. 

Focus group data were collected from 52 students during February and March of 1990. Five 
to nine students participated at each of six sites. The sites, distributed geographically across 
the state, were chosen to represent a variety of community college variable factors (urban, 
suburban, and rural), (traditional college transfer, traditional vocational education, and 
comprehensive community), (small, medium, and large), (high technology, smokestack, and 
agricultural), (local, regional, and international), and (geographically Eastern, Central, and 
Western Washington State). Members of the student focus group were chosen from students 
in academic, vocational preparation, vocational supplemental, and contractual classes who 
said they were in school to get a better job or to update their skills. 

The focus group sessions lasted from two to two-and-one-half hours in length. A list of open- 
ended questions with follow-up probes was used as a guide to conducting and moderating the 
focus group discussions. The group discussions were tape recorded and typed verbatim. In 
addition, each focus group member filled out a supplemental survey providing demographic 
and background information. 

Colleges provided Borton with the names and phone numbers of students who were in 
classes, which had a high probability of containing incumbent workers. 

She called persons on each of these lists to insure that there were at least 10 persons 
committed to participate in each group who met the qualifications. A telephone screen was 
used to assure that each informant met the qualifications. Persons chosen to be informants 
were sent letters with the details of the time, place, and a map or directions two weeks before 
the session. A phone call was made to each the day before the focus group to confirm their 
intentions to attend. It was assumed that 60-80 percent participation would be achieved so 
that there would be from 5-10 in each group. A total of 234 persons were called initially for 
the telephone screen and of that number 22 percent qualified and were able to participate. 

An attempt was made to avoid choosing students who were from the same classes or 
programs so as to increase the diversity of novel answers and perspectives in the focus 
groups. The researcher achieved the targeted participation 92 percent of the time. A gift 
certificate of $25.00 redeemable in the college bookstore was offered to the students as a 
token of appreciation for their participation. 

Borton developed three types of instruments for the focus group students. There was a 
preliminary multiple choice questionnaire used to probe and stimulate the thinking of the 
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informants about the topic. It consisted of 12 multiple choice questions. It took about 5-10 
minutes to administer and was completed at the beginning of the focus group sessions as 
people were arriving. 

A consent form was developed for each group which passed inspection of the University of 
Washington Human Subjects Review Committee. These forms were used to obtain 
permission to use the data collected and to assure anonymity. 

A focus group discussion guide was developed using information obtained from the literature 
review and of naturalistic observations. The guide consisted of five open-ended questions 
with several follow-up probes for each question. These procedures were used for expansion 
of the range of the answer; to obtain more specificity; to go deeper into the affective, 
cognitive, or evaluative meanings of the answer; and to describe the personal context from 
which they came. Sometimes probes were used to test the strength and robustness of the 
response. Questions gradually focused in on the topic. One hypothetical situation was 
presented to each group to encourage answers from a variety of dimensions, to avoid 
implying answers or ways to respond, and to encourage answers based on the informant’s 
own specific contexts. Brainstorming was used at the beginning of each focus group to obtain 
a breadth of responses, and projective techniques were used later in the group schedules to 
encourage depth in responses. 

Borton used a computer program called ethnograph to analyze the transcripted interviews and 
the transcripted debriefing session between herself and the focus group monitor. 
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STUDENT FOCUS GROUP DISCUSSION GUIDE 

Your participation in answering these questions is voluntary. You may choose not to answer 
any of them. 

1 . What is your current life situation and how did you happen to decide to return to school? 

(Use a chronological approach in telling how you decided to return to school.) 

Follow-up probes: 

a) What were: the positive contributing factors? Supports? 

The negative hindering factors? Barriers? 

b) How did you overcome or surmount the barriers? 

c) What sources of information did you use in reaching your decision? 

2. What is your evaluation of your previous experience with and performance in education and 
what are your current expectations and experience with education and your performance? 
Follow-up probes: 

a) Historical experience of education and of continuing education? 

b) Your orientation to learning? 

c) Evaluation of your self as a student? Your resources? Barriers you have to your learning 
progress? Compensations for them? 

d) What your are experiencing now and how you feel about it? 

e) What resources do you have that enable you to cope with school? 

3. What are your educational needs (needs for education), purposes in getting education, goals 
(outcomes aimed at), expected benefits, and rewards and incentives for getting further 
education? How will you know that you have achieved them? 

Follow-up probes: 

a) Has your employer expressed a need for you to obtain certain 
skills or upgrade skills? Are these things you want? 

b) What do hope for by by getting this education? 

c) How will it benefit you? 

d) What rewards or incentives that you are aware of are available? 

e) Does your employer offer you any of these and how? 

f) How will you evaluate whether you are achieving your goals? 

g) What criteria will you use? 

4. What is the attitude of important people toward education and toward your returning to school? 
Follow-up probes: 

a) Family members? 

b) Employers? 

c) Self? 

d) Faculty/Staff? 

e) Friends? 

5. How has it been for you to return to school? 

Follow-up probes: 

a) What are the problems you have faced? 

b) What are the joys and benefits you have experienced? 

c) What are the supports you have found that help and encourage your progress? 

d) How did you find them? 
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Student Questionnaire 
SAMPLE STUDENT QUESTIONNAIRE 



Your participation in answering these questions is voluntary. You may choose not to answer any of 
them. 



Student Number (Social Security Number) 



Please check the most appropriate answer. This questionnaire represents challenges some of which 
everyone faces at some time in their lives. We want to know what challenges are facing you as a JRT 
student. 



1. Highest level of school 
previously achieved 



( ) Attend High School 
( ) High School Grad 
( ) Apprenticeship 
( ) Attend Vocational 



School 



2. Age 



( ) Attend College 
( ) Associate’s Degree 
( ) Bachelor’s Degree 
( ) Master’s Degree 

( ) Under 21 
( ) 21-25 
( ) 26-35 
( ) 36-49 
( ) 50 or over 



3. Current Income Level 
a. Type 



( ) Individual 
( ) Family 



b. Level 



( ) No Income, Dependent 
( ) Under $10,000 
( ) $10,000-515,000 
( ) $15,000-520,000 
( ) $20,000-530,000 
( ) Over $30,000 



4. Life Situation 

a. Marital Status 

(Single includes divorded 
and never married here) 



( ) Married, With Children 
( ) Married, No Children 
( ) Single, With Children 
( ) Single, No Children 
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b. Living Situation 



( ) Living Alone 
( ) Living With Others, Not Family 
( ) Living With Parents 
( ) Living With Parents 
( ) Living With Family 
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4. Life Situation (Cont.) 

c. Responsibilities 



d. Crises Presently Faced 



5. Current Life Stage 



6. Current Career Stage 



7. How is Your Tuition Paid For 



8. Previous Experience With 
Educational Training 



( ) None 
( ) Self 
( ) 1-2 Others 

( ) 3-4 Others 
( ) Over 5 

( ) Marriage 

( ) Divorce 

( ) Loss of Job 

( ) Underemployment 

( ) With Children 

( ) Empty Nest 

( ) Retirement 

( ) Change of Job 

( ) Loss of Self-Esteem 

( ) Experiencing Physical Changes 

(Menopause or Loss of Health, etc.) 

( ) Ailing Parents 
( ) Death of Family Member/Friend 
( ) Other 

( ) Single, Never Married, No Children 
( ) Newly Married/Coupled 
( ) Parent, Young Children 
( ) Parent, School-Aged Children 
( ) Parent, Teenage Children 
( ) Elderly Parents 
( ) Senior Citizen 

( ) No Career, Still Searching 

( ) Worked Entry-Level Job 

( ) Skilled and Experienced Worker/Technician 

( ) Supervisor 

( ) Mid-Level Management 

( ) Top Management 

( ) CEO 

( ) Business Owner 
( ) Self 

( ) Job Pays Tuition Reimbursement, Specify 

Conditions of Reimbursement 

( ) Student Loan 
( ) Scholarship, Grant 
( ) Work Study 
( ) Other, Specify 

( ) On-the-Job Training 
( ) “In House” Classes, Workshops, 

Seminars, and/or Orientations 
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8. Previous Experience With 


( ) Apprenticeship 


Educational Training 


( ) College Courses (Vocational or 


(Cont.) 


College Transfer Courses) 

( ) College Workshops, Seminars 
( ) Professional Association Workshops, Seminars 
( ) Television or Telecommunication Classes 
( ) Correspondence 
( 1 Specifv 


9. Do You Intend to 


( ) Stay in Your Current Job 
( ) Stay in Your Current Career 
( ) Stay With the Same Company 
( ) Work for Certification 




( ) Explore Possible New Careers 
( ) Change Jobs 
( ) Change Careers 



10. Which of the following services are available to you at school and which do you use? 





Available 


Use 


Child Care 


( 


) 


( 


) 


Transporation Reimbursement 


( 


) 


( 


) 


Financial Aide 


( 


) 


( 


) 


Career Counseling 


( 


) 


( 


) 


Personal Counseling, Individual 


( 


) 


( 


) 


Personal Counseling, Group 


( 


) 


( 


) 


Skill Assessment 


( 


) 


( 


) 


Tutoring 


( 


) 


( 


) 


Job Placement 


. ( 


) 


( 


) 


Representation in Student Government 


( 


) 


( 


) 


Family Activities 


( 


) 


( 


) 



11. How would you classify yourself 



( ) Lifelong Learner, Continually Seeking New Classes 
( ) Periodically Returning to Education 
( ) Using Education to Explore Change 
( ) Using Education to Solve a Life Crisis 



12. I learned about the opportunities 
for returning to from a: 



( ) Counselor 
( ) Brochure 
( ) Mailed Class Schedule 
( ) Friend 
( ) Counselor 
( ) College Recruiter 
( ) Career Fair 
( ) Poster, Billboard or Ad 
( ) College Catalog 
( ) Other 
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NOTICE 

REPRODUCTION BASIS 




This document is covered by a signed “Reproduction Release 
(Blanket)” form (on file within the ERIC system), encompassing all 
or classes of documents from its source organization and, therefore, 
does not require a “Specific Document” Release form. 




This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may 
be reproduced by ERIC without a signed Reproduction Release 
form (either “Specific Document” or “Blanket”). 




